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Executive Summary 


The NASA/Navy Benchmarking Exchange (NNBE) was undertaken to identify practices and procedures 
and to share lessons learned in the Navy’s submarine and NASA’s human space flight programs. The 
NNBE focus is on safety and mission assurance policies, processes, accountability, and control measures. 
This report is an interim summary of activity conducted through October 2002, and it coincides with 
completion of the first phase of a two-phase fact-finding effort. 

In August 2002, a team was formed, co-chaired by senior representatives from the NASA Office of 
Safety and Mission Assurance and the NAVSEA 92Q Submarine Safety and Quality Assurance Division. 
The team closely examined the two elements of submarine safety (SUBSAEE) certification: 1.) new 
design/construction (initial certification) and 2.) maintenance and modernization (sustaining 
certification), with a focus on: 1.) Management and Organization, 2.) Safety Requirements (technical and 
administrative), 3.) Implementation Processes, 4.) Compliance Verification Processes, and 5.) 
Certification Processes. 

The overall schedule and agenda supported early NASA benchmarking of the Navy. The activity to date 
included numerous meetings at the Washington Navy Yard (WNY), as well as trips to Groton, CT, and 
Portsmouth Naval Shipyard (PNS) in Kittery, ME. In addition, the NAVSEA team was hosted at the 
NASA Aerospace Safety Advisory Panel (ASAP) meetings at Johnson Space Center, Houston, TX, 
November 5-7, 2002. 

NASA’s examination resulted in identification of the following key attributes of the Navy submarine 
safety program: 

- Safety requirements are clearly documented and achievable, with minimal use of waivers or tailoring. 

- Requirements are implemented through capable and carefully controlled processes. 

- Compliance is verified through rigorous in-line management control and assurance processes. 

- Compliance is also verified independently of program management by a separate compliance 
verification organization. 

- A strong safety culture exists with emphasis (training and awareness) on understanding and learning 
from past failures (e.g., USS THRESHER). 

- A centralized technical authority and a large operational experience base facilitate continuous 
learning and improvement in design, manufacturing, and operations. 

Potential opportunities for NASA to consider, based on observed Navy practices, are listed below and 
discussed in Section 4 of this report: 

- Development of functional safety requirements for future human rated space systems. 

- Implementation of the NAVSEA organizational model for compliance verification. 

- Examination of the proposed compliance verification approach for Orbital Space Plane. 

- Expansion and management restructuring of lessons learned within NASA. 

- Incorporation of noted NASA and Navy program failure case studies in lessons learned training. 

- Consideration of the NAVSEA knowledge retention approach (increased ETE ceiling authorization 
for hiring) as part of the Administrator’s workforce management initiative. 

During the remaining portion of this activity, the NASA team will participate as observers during a 
SUBSAEE certification audit and conduct follow-on discussions concerning Naval Reactors safety 
processes, quality assurance, software, and human factors. NASA will host the Navy team at Kennedy 
Space Center (KSC), the Michoud Assembly Eacility (MAE) in New Orleans, LA, Stennis Space Center, 
MS, and Thiokol in Ogden, UT. Separate subject matter expert trips are planned to benchmark software 
development and assurance, and human factors. The NNBE is expected to be completed by May-June of 
2003, followed by a final joint NASA/Navy report in the August-September 2003 time frame. 
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NASA/NAVY Benchmarking Exchange 
(NNBE) Interim Report 

1.0 Introduction 



1.1 Motivation 

Notable similarities exist between human space flight and nuclear submarine programs. 
Both spacecraft and submarines operate in extreme environments, both require 
integration of complex systems and subsystems, and both must maintain the highest 
levels of safety and reliability in order to perform their missions. The Navy has 
continued to operate safely and effectively in resource-constrained and declining- 
production environments. Furthermore, as NASA explores the application of nuclear 
propulsion and power for space exploration, lessons learned from the Navy’s nuclear 
safety program could be beneficial. Thus, given the current management challenges 
NASA is facing, the Agency might benefit from an in-depth examination of the 
engineering management, safety, and mission assurance (SMA)^ practices employed by 
the Navy submarine force. 

The design, test, operation, and maintenance of submarine and human space flight 
programs have many attributes in common: 

- Missions are of national importance, 

- Safety is essential - severe consequences are associated with critical system failure, 

- Operating environments are inherently hazardous (hostile), 

- Nuclear energy systems are currently in- use/are planned for expanded use, 

- People and the surrounding environment need to be protected from hazards, 

- Complex, tightly coupled systems are involved, 

- People play a critical role in complex processes, 

- System requirements include highly reliable, long-term operations, 

- Mission models include both new design/construction and ongoing/sustained 
operation of fleets, 

- Operational integrity must be sustained throughout maintenance/modernization cycles, 

- Operational integrity also must be sustained throughout management challenges such 
as downsizing, production decline, budget constraints, and workforce instabilities, 

- Parts obsolescence is an ongoing challenge, 

- New material applications continue to push the envelope, 

- Modular components provided by multiple vendors present a systems engineering 
challenge, 

- NASA and the Navy operate under Federal procurement regulations, and 

- NASA and the Navy were both directed to have an independent compliance 
assurance program and organization for the safety of space and submarine operations, 
respectively. 


^ In NASA parlance SMA includes safety, reliability, maintainability, and quality assurance. 
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It should also be noted that there are important differences to be aware of when 
comparing NAVSEA and NASA programs and operations. Submarine and human space 
flight programs have different: 

- Degrees of reliance on evolution and legacy of design, 

- Opportunities for intervention and recovery from critical failures, 

- Weight and volume criticalities, 

- Cultural environments, 

- Technical responsibility, authority, and accountability, 

- Safety and mission assurance requirements management, 

- Verification and compliance approaches, 

- Program management, organization, and integration of safety and mission 
assurance activities, and 

- Design approaches (deterministic versus probabilistic). 

These similarities and differences are discussed in further detail (section 4.0) as context 
for the key observations and opportunities provided in this benchmarking exchange. 

Navy submarines and NASA’s human space flight programs have, in general, 
demonstrated outstanding safety records over sustained time periods. However, both 
Navy and NASA operational experience has been punctuated by a few significant failure 
events [e.g., USS THRESHER (SSN-593), USS SCORPION (SSN-589), Apollo- 1 pad 
fire, STS -5 IE Challenger] that have led to increased focus on safety. 



Figure 1.1 USS THRESHER (SSN 593). Figure 1.2 STS-51L Challenger. 

In August 2002, NASA began a two-way benchmarking exchange exploring life-cycle 
safety management paradigms, including a rigorous examination of relevant 
organizational structure, risk management processes, safety policies and procedures, 
work processes, tools, techniques, verification and compliance processes, and 
certification processes. The initiating letter from the NASA Administrator to the 
Secretary of the Navy is shown in Appendix A. 

1.2 Method 

Management Direction and Eramework for Benchmarking 

A management team was formed to direct the overall NNBE. This team is co- chaired by 
senior representatives from the NASA Office of Safety and Mission Assurance and the 
NAVSEA 92Q Submarine Safety and Quality Assurance Division. The NNBE 
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Management Team developed a common "lens for analysis" to address the overall goals 
set forth in the Administrator’s letter. The exchange is looking at new 
design/construction (initial certification) and maintenance/modernization (sustaining 
certification). Five topics are being examined in detail: 

1) Management and Organization, 

2) Safety Requirements (technical and administrative), 

3) Implementation Processes, 

4) Compliance Verification Processes, and 

5) Certification Processes. 

Within this framework the Submarine Safety (SUBSAFE) Program represented a major 
area of interest and focus. Additionally, the Naval Nuclear Propulsion Program safety and 
assurance activities are being examined within the context of new design/construction 
and maintenance/modemization. 

Approach 

The NNBE Management Team identified a specific data gathering approach that includes 
the following areas and activities: 1.) document exchange, 2.) Navy-hosted information 
exchange events, 3.) NASA- hosted information exchange events, 4.) subject matter 
expert technical exchange meetings (e.g., software, quality assurance, human factors, 
etc.), and 5.) on-line collaboration (eGov). 

Exchange Activity To Date 

As of the date of this interim report, the NNBE Management Team has met six times 
beginning with the kickoff meeting at the Washington Navy Yard (WNY) on August 13- 
15, 2002. The overall schedule and agenda was front- loaded with Navy- hosted events to 
support early NASA benchmarking. Initial NASA benchmarking team meetings at the 
WNY covered NAVSEA Headquarters’ roles in both new design/construction and 
maintenance/modernization. Erom September 17- 19, 2002, NASA observed the 
SUBSAEE Working Group meetings in Washington, DC. NAVSEA 08 Naval Reactors 
gave the NASA team an overview briefing on October 1, 2002, and NASA and NAVSEA 
08 are in the process of following up on interest items identified in the initial meeting. 

The NASA team went to Groton, CT, to observe NAVSEA conduct SUB SAFE 
functional audits of the Supervisor of Shipbuilding - Groton and Electric Boat 
Corporation (September 23-27, 2002). The team returned for examination of new 
design/construction activities at Electric Boat on October 16-18, 2002. 

Erom October 21-23, 2002, the NASA team reviewed submarine engineering, 
maintenance planning, and logistics activities at Portsmouth Naval Shipyard (PNS) in 
Kittery, ME. The site visit also included discussions with Submarine Maintenance 
Engineering, Planning and Procurement (SUBMEPP) and Ship Availability Planning and 
Engineering Center (SHAPEC). Einally, the NASA team met with the Naval Sea 
Eogistics Center Detachment in Portsmouth, NH. 
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To kickoff the NASA-hosted events, the NAVSEA benchmarking team was hosted at the 
NASA Aerospace Safety Advisory Panel (ASAP) meetings at Johnson Space Center, 
Houston, TX, November 5-7, 2002. 

Next Steps - NASA Team Phase-2 (January - June 2003) 

During phase- 2, the NASA team will complete its assessment activity by participating as 
observers during a certification audit of the USS ASHEVILLE that is scheduled to occur 
in Eebruary 2003, at Puget Sound Naval Shipyard, Bremerton, WA. In addition, follow- 
on discussions are proposed for the following areas: 

Naval Reactors Safety Processes: details of Naval Reactors safety, quality assurance, 
and new-design methodologies and processes. 

Quality Assurance: collaboration opportunities in the areas of supplier risk 
evaluation, workmanship criteria, and corrective action approaches. 

Software: exchange of approaches for requirements definition and flow-down, and 
approaches for implementation of software verification and validation processes. 

Human Eactors: collaboration opportunity with NAVSEA 03 (Human Systems 
Integration in System Design) to evaluate and develop possible human/system 
interface technical standards, policies, and processes. 

Specialized Technical Discussions: Expert- to- expert technical interchange meetings 
in specialized technical areas such as software, quality assurance, human factors, 
supplier audit, and other specific technologies (e.g., explosive bolt pyro- technology). 

Next Steps - Navy Team Phase- 2 (January - June 2003) 

The Navy is currently in the process of identifying additional items for further 
investigation based on attending the ASAP meetings conducted at JSC. 

During phase- 2, NASA will host the Navy team at Kennedy Space Center (KSC) during 
the first week of January and again during the last week of Eebruary (tentative dates). 
KSC events will include a Elight Readiness Review (ERR), work authorization, and 
implementation, work control and review processes, critical process management, and 
human factors discussions. Other trips are planned to the Michoud Assembly Eacility 
(MAE), in New Orleans, LA, Stennis Space Center, MS, and Thiokol in Ogden, UT. 

Each trip will address multiple topics and processes. In addition, separate subject matter 
expert trips are planned to benchmark software development and assurance, and human 
factors. 
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Next Steps - Continued On- Line Collaboration 


The on-line collaboration (eGov) of NAVSEA experts with NASA counterparts has 
begun and is anticipated to expand throughout phase- 2 of the NNBE. It is anticipated 
that there will be long-term collaboration among Navy and NASA subject matter experts 
using NASA PBMA-KMS work groups. 

Report Organization 

Eollowing this section, section 2 provides background orientation for the reader 
concerning key organizations within the NAVSEA submarine domain. Also included is a 
high level overview of submarine hazards and the organizations and processes in place to 
mitigate those hazards. Section 3 contains detailed summaries and key observations 
developed by the NASA team based on Navy- hosted benchmarking events conducted 
through October 31, 2002. In section 4, the NASA team describes potential opportunities 
for the Agency to build upon selected NAVSEA processes and best practices. 

1.3 Scope 

Table 1.1 presents a comparison between the NASA NNBE mission objectives, 

(contained in the Administrator's letter and attachment) and the accomplishments to date. 


Table 1.1 


INTERIM ACCOMPLISHMENT STATUS 

Excerpts from O'Keefe Letter 

Status 

Benchmark Navy nuclear submarine program 
with focus on safety and mission assurance 
policies, processes, accountability, and control 
measures that can assist NASA in meeting current 
human space flight challenges. 

Largely accomplished. Gained an 
understanding of the Navy-side of the 
benchmark equation. While an 
understanding/documentation of 
NASA's current SMA processes 
exist, a formal, direct comparison has 
not been conducted in this report. 

Develop a set of “lessons learned” that could be 
effectively applied to NASA’s human space flight 
programs. 

Potential opportunities identified. See 
section 4. 

Understand the nuclear Navy top-down approach 
for ensuring accountability and control in safety 
critical areas. Understand the floxwdown of top- 
level safety philosophy and requirements. 

Accomplished. Addressed in section 
2.2 and section 3.1. 

Understand the overarching risk management 
posture or logic employed in making decisions 
concerning competing and often conflicting 
program dimensions of cost control, schedule, 
mission capability, and safety. 

Accomplished. Multiple discussions 
in section 3. 
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INTERIM ACCOMPLISHMENT STATUS 

Excerpts from O'Keefe Letter 

Status 

Understand the cultural dimensions of the 
SUBSAFE Program. 

Accomplished. See sections 1.3, and 
4. 

Acquire a process- level understanding of safety, 
risk management, and mission assurance 
processes within the Navy nuclear submarine 
SUBSAFE Program. 

Accomplished for SUBSAFE 
Program. See section 3.1. 

Additional data acquisition in 
progress for NAVSEA 08 portion of 
safety, risk management, and mission 
assurance processes. 

Understand the policies, procedures, practices, 
and processes for 1.) definition of requirements, 

2. ) verification of ongoing implementation, and 

3. ) certification of operational readiness. Clearly 
identify organizational accountability and 
responsibilities for SUB SAFE implementation. 

Accomplished. See section 3.1 and 

3.3. 

Understand the authority and relationships 
between various control boards and processes 
responsible for hardware design, software design, 
manufacturing processes, test/certification, and 
operations. 

Accomplished. See section 3. 
NAVSEA technical design processes 
examined for the USS VIRGINIA 
and SSGN Conversion programs. 

See section 3.2 and 3.3. 

Understand the role of systems engineering 
authority within the Trident program for 
integrating and managing change. 

NAVSEA systems engineering roles 
examined specifically for USS 
VIRGINIA and SSGN Conversion 
programs. (Trident processes are 
similar.) See section 3.2. 

Understand the organizational responsibilities and 
processes employed for configuration 
management of requirements baselines. 

Accomplished. NAVSEA systems 
engineering roles identified 
specifically for USS VIRGINIA and 
SSGN Conversion programs. See 
section 3.2. 

Explore and understand the policies, processes, 
and controls implemented for assuring the 
ongoing capability of people in critical processes. 

Accomplished. Examined NAVSEA 
and EB approaches to management 
control during downsizing. See 
section 3.2.5. 

Need to understand approaches for ensuring a 
reliable (human reliability) and capable workforce 
in terms of health, stress, overtime, extended 
duty, physical, and psychological work 
environment. 

Not yet fully explored. Additional 
information to be acquired in phase- 
2. See discussion (Next Steps - 
Phase- 2) above. 

Need to understand the administrative or 
management approaches implemented to ensure 
that critical processes maintain necessary staffing 
levels to function in a stable, capable, and 
controlled manner. 

Accomplished. Examined NAVSEA 
and EB approaches to management 
control during downsizing. See 
section 3.2.5. 
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INTERIM ACCOMPLISHMENT STATUS 

Excerpts from O'Keefe Letter 

Status 

... and maintaining a skilled and motivated 
workforce in the face of budget and schedule 
pressures - much as was experienced by the 
nuclear submarine program during a downturn in 
production in the early 1990’s. 

Accomplished. Examined NAVSEA 
and EB approaches to management 
control during downsizing. See 
section 3.2.5. 

. . . related to the critical issues facing NASA of 
maintaining product quality and safety, 
accomplishing required performance and safety 
upgrades. 

Partially understood. Additional 
work in phase-2. See section 3.4.4. 

. . . furthermore, as NASA explores the application 
of nuclear propulsion and power for space 
exploration, lessons learned from the Navy 
nuclear safety program could be beneficial. 

Partially understood. Additional 
work in phase- 2. 
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2.0 Background 

2.1 Orientation 

Naval Sea Systems Command (NAVSEA), largest of the Navy's five systems eommands, 
is loeated at the Washington Navy Yard, in Washington, DC. NAVSEA is the 
organization responsible for designing, aequiring, maintaining, and modernizing ships 
and systems for the Navy in aeeordanee with Eleet and Offiee of Chief of Naval 
Operations (OPNAV) Sponsor needs and requirements. The Program Exeeutive Offieer 
for Submarines (PEO SUB) is responsible for aequisition of new submarines and 
systems. The Deputy Commander for Submarines (NAVSEA 92) is responsible for 
support of in-service submarines and systems. Currently, RADM Phil Davis is “doubled 
hatted” in these roles. Eigure 2.1 shows the overall NAVSEA organizational chart as of 
October 31, 2002. ^ 

Two commercial firms currently build submarines: General Dynamics Electric Boat 
Corporation in Groton, CT, and Northrop Grumman Newport News Shipbuilding, located 
in Newport News, VA. 

Submarine maintenance and modernization occurs at a number of different locations 
around the world. Major work is typically performed at Naval (i.e., government) 
shipyards. Today there are four operational Naval shipyards: Portsmouth Naval 
Shipyard, Kittery, ME; Norfolk Naval Shipyard, Portsmouth, VA; Puget Sound Naval 
Shipyard, Bremerton, WA; and Pearl Harbor Naval Shipyard & Intermediate 
Maintenance Eacility, Honolulu, HI. 

Key NAVSEA organizations referred to in this report are identified below: 

NAVSEA 92: Submarine Directorate 

NAVSEA 92Q: SUBSAEE/Quality Assurance 

NAVSEA 92T: Technical 

NAVSEA 08: Naval Reactors 

NAVSEA 05: Ship Design, Integration and Engineering 

NAVSEA 04: Eogistics, Maintenance & Industrial Operations 

NAVSEA 03: Human Systems Integration in System Design 

PMS: Program Managers (e.g., PMS-398 is the Program Manager for the 

SSGN Ohio Class Conversion) 

2.2 Submarine Hazards and Organizations That Manage Them 

Submarine hazards are identified, and risks are mitigated, throughout all levels of the 
Navy. Eigure 2.2 identifies major high-level hazard categories and the organizational 
responsibilities for mitigation and management of the hazards within NAVSEA. 

^ At the time this report is being prepared NAVSEA is undergoing a reorganization that will result in 
changes in the organizational designators. 
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While this report explores many of the submarine hazard domains, it does not investigate 
the subjects of fire or weapons safety. 


One management practice employed by NAVSEA is the use of boundaries to bring focus 
to specific categories of hazards. For example, SUBSAFE Boundary Books'^ are used to 
identify components and systems that could play a role in a flooding incident. The 
paragraphs below outline the governing requirements documents, organizational 
boundaries, technical authority, compliance verification responsibilities, and key 
assurance processes associated with the Navy submarine program. 

It is important to note that each of the NAVSEA organizations shown in figure 2.2, and 
discussed below, performs its duties as a support and control organization, ultimately 
serving the submarine life- cycle manager (the program manager) and the submarine 
operator (the Fleet Type Commander). 

2.2.1 Flooding Prevention and Ability To Recover From Flooding 

The submarine safety program technical and administrative requirements identified in the 
SUBSAFE Manual^ are designed to provide maximum reasonable assurance that 1.) 
seawater is kept out of the submarine (i.e., uncontrolled flooding does not occur), and 2.) 
the submarine can recover from a flooding casualty (e.g., loss of depth control). 

This is achieved through initial SUBSAFE certification of each submarine prior to final 
delivery to the Navy, and by sustaining SUBSAFE certification throughout the life of the 
submarine. The NAVSEA 05 organization is the final authority for the technical 
requirements contained in the SUBSAFE Manual. Implementation of the SUBSAFE 
requirements is the responsibility of all persons and organizations involved in the 
submarine’s life cycle: designers, program managers, program technical directors, 
submarine operators, and organizations involved in the acquisition, supply, logistics 
support, maintenance, refit, modernization, and overhaul of submarines. 

NAVSEA 92Q has overall responsibility for overseeing the SUBSAFE Program and 
verifying compliance with its requirements. At each facility performing SUBSAFE 
work, a local SUBSAFE Program Director (SSPD) provides oversight for work at that 
facility and is responsible for independently verifying compliance with SUBSAFE 
requirements. NAVSEA 92Q audits policies, procedures and practices at each facility as 
well as the effectiveness of the oversight provided by the local SSPD. Key processes and 
practices used in administering the SUBSAFE Program include SUBSAFE functional 
audits, SUBSAFE certification audits, the Re-entry Control process (configuration 
management), the URO-MRC process (mandatory inspections and tests), annual 
SUBSAFE training, and the use of SUBSAFE Boundary Books. These topics are 
discussed in greater detail in section 3.1. 


e.g., NAVSEA S9SSB-X9-SCB-010/(N) SSCB, Submarine Safety Certification Boundary Book - 
SSBN726 Class. 

^ NAVSEA 0924-062-0010, "Submarine Safety (SUBSAEE) Requirements Manual.” 
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2.2.2 Deep Submergence System Crew Safety 

The Deep Submergence System (DSS) program scope includes deep submergence 
research vehicles (e.g., Alvin) and diver lockout and dry deck facilities that may be 
incorporated in a submarine. DSS Scope of Certification (SOC) requirements are set out 
in the DSS- SOC Manual.^ As described in the manual, the intent is . .to provide 
maximum reasonable assurance that a material or procedural failure that imperils the 
operators or occupants will not occur. Certification ensures that conditions adversely 
affecting the safety of DSS personnel are mitigated and verifies that no recognizable 
unsafe conditions exist. The certification process also establishes maximum reasonable 
assurance that DSS personnel may be recovered without injury if there is an accident.” 

Key areas of focus include hazards associated with explosion, implosion, off- gassing of 
materials, electrical systems, and the quality of breathing air. The technical authority for 
DSS-SOC requirements resides with NAVSEA 05. Implementation of the DSS-SOC 
requirements is the responsibility of designers, program managers, program technical 
directors, operators, and organizations involved in the maintenance/modemization of 
both submarines and deep submergence vehicles. NAVSEA 92Q and the facility-based 
SUBSAEE Program Directors are responsible for independently verifying compliance 
with the DSS-SOC requirements. 

2.2.3 Systems Safety and Hazard Analysis 

It is important to note that the ship specification or "ship spec" for each new class of 
submarine is built upon the evolution of ship specs from previous class designs, updated 
for war fighting and mission capability, technical advances, and lessons learned. 

This reflects the continuous process of risk identification and mitigation through 
successive design, build, and operate cycles. Both “best practices” and “lessons learned” 
are captured and included in the next revision of the ship specs and sub -tier requirements 
documents to provide continual process improvement in submarine platform 
requirements. 

Eor new design programs, such as the VIRGINIA Class submarine or the OHIO Class 
submarine conversion to SSGN, general systems hazards (excluding flooding, DSS, 
weapons, and nuclear propulsion) are addressed through the safety processes found in 
Military Standard 882D.^ Responsibilities reside with new submarine program managers, 
their design (technical) directors, and the subordinate system safety integrated product 
team (IPT) leads. Technical risk management in the design process addresses safety 
assurance. Eormal programmatic risk management is conducted for new class design 


® NAVSEA SS800-AG-MAN-010/P-9290, "System Certification Procedures and Criteria Manual for Deep 
Submergence Systems.” 

’ MIL-STD-882D - Department of Defense Standard Practice for System Safety, February 10, 2000. 
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Q 

systems in accordance with DoD 5000. 2-R. However, the focus is primarily on 
programmatic risks as opposed to safety risk management because mitigation of safety 
risks has been effectively addressed in the ship specs which capture successive 
generations of “design, build, and operate” lessons learned, as mentioned above. 

Within a given program management team, key systems safety processes include: the 
system safety IPT, Material Safety Review Panel, Risk Management IPT, Software 
Safety IPT, and the Safety Certification Process Action Team. Key planning documents 
include: System Safety Plan, and the Material Selection Requirements Plan (MSRP). 

2.2.4 Weapons Safety 

Weapons safety is not a focus of this benchmarking assessment; however, the following 
synopsis is provided for completeness. Explosives safety rules and regulations include 
various NAVSEA ordnance manuals and guidance documents. Technical authority for 
weapons safety requirements resides within the NAVSEA 05 organization. Within a 
NAVSEA design team, implementation of the weapons safety requirements is the 
responsibility of program managers, program technical directors, and weapons IPT leads. 
The key certifying body is the Weapons System Explosives Safety Review Board 
(WSESRB). Key organizations include the Weapons System Integration Team (SIT) and 
the Software Safety Technical Review Panel, which implement the formal weapons 
certification process. 

2.2.5 Reactor Safety 

NAVSEA 08 is a joint Navy /Department of Energy organization that is responsible for 
all aspects of Navy nuclear propulsion, including research, design, construction, testing, 
training, operation, maintenance, and the ultimate disposition of nuclear propulsion 
plants. Reactor safety is fundamentally addressed in each aspect of NAVSEA OS’s 
responsibilities. Eor example, propulsion plant design features include inherent self- 
regulation for stability, equipment redundancy, and rugged design for battle shock. As 
another example, the nuclear propulsion plants are operated and maintained by highly 
trained crews, who receive over a year of academic and hands-on training before 
qualification. Subsequently, operators receive periodic training to maintain their 
proficiency. A summary of NAVSEA OS’s practices regarding reactor safety can be 
found in Admiral Hyman G. Rickover’s (the founder and director of NAVSEA 08 for 34 
years) testimony to Congress on May 24, 1979, following the reactor accident at Three 
Mile Island. 

2.2.6 Manufacturing/Construction/Integration Compliance 

NAVSEA 04, including the Naval Sea Eogistics Centers, provides policy guidance and 
technical leadership in areas concerning industrial operations, logistics, quality assurance, 
material control, and supplier control. NAVSEA 04 manages the operation of the Naval 


DoD 5000. 2-R- “Mandatory Procedures for Major Defense Acquisition Programs (MDAPS) and Major 
Automated Information System (MAIS) Acquisition Programs,’’ 10 June 2001. 
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shipyards and directly manages quality assurance activities and material control. Work 
performed at Naval shipyards is predominately maintenance, overhaul, and 
modernization. The Supervisor of Shipbuilding, also managed by NAVSEA 04, oversees 
contract performance in private shipyards such as Electric Boat and Newport News 
Shipbuilding, where construction, maintenance, and modernization work are performed. 
Key overarching quality assurance requirements documents are defined by activity and 
include: 

Naval Shipyard Quality Program Manual, ^ 

SUPSHIPs Operations Manual, Chapter 11, Contract Quality Assurance Program, 
Quality Program Requirements ^ ^ 

1 9 

Quality Management Systems - Requirements 

1 o 

Joint Elect Maintenance Manual, Volume V, Quality Maintenance 

The quality assurance organizations of the various activities play a key role in validating 
compliance with SUBSAEE Program requirements and in compiling the objective quality 
evidence or OQE necessary to support SUBSAEE and/or DSS-SOC certification. 


® NAVSEA -TL855-AA-STD-010, Naval Shipyard Quality Program Manual. 

S0300-B2-MAN-01A, SUPSHIPs Operations Manual, Chapter 11, Contract Quality Assurance Program. 
** MIL-Q-9858A, Quality Program Requirements. 

ISO 9001-2000 Quality Management Systems - Requirements. 

CINCLANTFLT/CINCPACFLTINST 4790.3, Joint Fleet Maintenance Manual, Volume V, Quality 
Maintenance. 
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3.0 Summaries and Key Observations 


Building on the benchmarking framework and approach described in section 1, the basic 
elements of the submarine Navy’s safety and mission assurance function has been 
separated into the following seven topic areas: 

3.1 SUBSAFE Program 

3.2 Management and Organization 

3.3 Safety Requirements 

3.4 Implementation Processes 

3.5 Compliance VerificationAVork Review Processes 

3.6 Compliance Certification Processes 

3.7 Naval Reactors 

The purpose of this sectbn is to document all factual information gathered during the site 
visits. Discussions are in the form of summaries, and topics or elements may be further 
expanded into sub-topic or sub-element descriptions. Key observations are provided 
following each element and/or sub-element. Given the great depth of detail presented 
here, the reader may wish to scan the composite of all key observations tabulated in 
Appendix B, and refer to Section 3 when further detail is required. 

3.1 SUBSAFE Program 

The fundamental purpose of the SUB SAFE Program is to provide maximum reasonable 
assurance that 1.) seawater is kept out of the submarine (i.e., uncontrolled flooding does 
not occur), and 2.) the submarine can recover from a flooding casualty (e.g., loss of depth 
control). Figure 3.1 depicts the development and key events in the evolution of the 
SUBSAFE Program. 

The SUBSAFE Manual identifies the technical and administrative submarine safety 
certification criteria that must be satisfied to permit NAVSEA’s initial certification of the 
submarine, and their recommendation for unrestricted operations, as well as the technical 
and administrative requirements that must be met during the ship’s operational life to 
maintain the certification. 

The SUBSAFE Program is a program of work discipline, material control, and 
documentation founded on solid technical requirements. 

Work Discipline 

Work discipline implies a knowledge of the requirements and compliance with those 
requirements, not just for the tradesman on the deck plates working with the hardware, 
but everyone who performs any kind of work associated with submarines. Individuals 
have a responsibility to know which, if any, SUBSAFE requirements pertain to their 
work and to comply with those requirements. 
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SUBSAFE Program Timeline 


NAVSEA 
SUB SAFE Program 
established June 
1963; 

Submarine Safety 
Certification 
Criterion established 
December, 20 1963. 
(Governing 
document.) 


i A 


Congressional 
Joint Committee 
on Atomic Energy 
Hearing 1963-64. 


A 


SUBSAEE 

Manual 

(NAVSEA 

0924-062- 

0010) 

published 

12/20/74. 

Became new 

governing 

document. 


A 


Revision of 
SUBSAEE Manual 
(NAVSEA 0924-062- 
0010) initiated 1986. 


SEA 92Q 
Sub QA 
Division 
established 
9/85 to 
conduct 
pre-Sea 
Trial 

SUBSAEE 

audits. 


NAVSEA 
SUBSAEE Senior 
Review Group 
1986-1988. 
(Recent shipyard 
incidents indicated a 
loss of safety focus.) 


Downturn in 
submarine 
production from 
5 subs/year to 
1/2 sub/year. 


AAAAA A 


USS THRESHER USS SCOP 

(SSN-593) (SSN-51 

4/10/63 5/22/6 

,PION 
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Challenger.. USS BONEE 

CTS SIT ■■■■' (SSN 583)F 

/ 4/24/8* 

Chernobyl 
Nuclear 
Reactor 
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1960s 
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* NAVSEA used these events to re- focus the SUBSAFE Program. 


Figure 3.1 
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Material Control 


Material Control involves everything required to ensure that correct material is installed 
correctly. It begins with ensuring the contracts that purchase material invoke appropriate 
requirements, all the way through receipt inspection, marking, storage, handling, and 
finally, installation in the submarine. The installing activity (i.e., field organization) is 
ultimately held responsible for ensuring that correct material is installed correctly. 

Documentation 

SUBSAFE documentation is in two forms, i.e., 1.) Selected Record Drawings & Data, 
and 2.) Objective Quality Evidence (OQE). Selected Record Drawings & Data are 
created when the submarine is designed and consists of such things as system diagrams, 
SUBSAEE mapping drawings, SUBSAEE Certification Boundary Books, Ship Systems 
Manuals, etc. These selected records are kept current throughout the life of the 
submarine. Objective Quality Evidence is created when work is performed (either new 
construction or overhauEmaintenance) and consists of such things as weld records, 
nondestructive evaluation (NDE) records, mechanical assembly records, test forms and 
technical work documents (in which data is recorded), and waivers & deviations, etc. 

Technical Requirements 

Specific requirements are defined for the ship’s pressure hull and supporting structure, 
systems and components either containing or exposed to seawater, and systems critical to 
operability to recover from a flooding casualty. These technical requirements identify 
system boundaries, specified fabrication, NDE, and material control performed. Also 
included are approval authority requirements for changes and departures from approved 
designs. The technical requirements are invoked by the ship specification to trigger the 
necessary processes used by the shipbuilder/maintenance activity. These processes result 
in compilation of the required OQE to achieve and maintain certification. OQE is any 
statement of fact, either quantitative or qualitative, pertaining to the quality of a product 
or service based on verifiable observations, measurements, or tests. 


Three elements are used to sustain 
SUBSAEE certification throughout the 
life of a submarine (see figure 3.2): 

1. ) the Re-entry Control Process, 

2. ) the Unrestricted Operations/ 
Maintenance Requirements Cards 
(URO/MRC) Program, and 3) Audits 
(Eunctional and Certification). 
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Re-entry Control Process 


This process is used to control work within the SUBSAFE boundary after the component, 
system, or submarine has been certified to comply with SUBSAFE requirements. Work 
that actually or potentially changes the approved configuration of any part of the 
SUBSAFE certification boundary is performed under re-entry control. OQE is used to 
verify that: 

Work was authorized and planned. 

Work was accomplished in accordance with specified instructions. 

Documentation and certification for the work has been reviewed for completeness by 
an independent party. 

Testing documentation has been reviewed for accuracy and completeness, and 
All certifications related to the re-entry control process have been reviewed for 
correctness and verified to be complete before the re-entry is closed. 

Re-entry control is a tool that helps maintain work discipline, establish personal 
accountability, and collect objective quality evidence. The purpose of re-entry control is 
to provide maximum reasonable assurance that areas disturbed within the SUBSAFE 
boundary are restored to a fully certified condition. 

URO/MRC Program 

This program provides the minimum requirements for maintenance of submarine 
certification for continued unrestricted operation to authorized test depth. The program 
assures continued acceptability of the material condition of the submarine hull integrity 
boundary, and of those items that affect recoverability by the accomplishment of periodic 
inspections and test and/or maintenance. The URO/MRC’ s provide specific requirements 
as to when, where, and how a test or inspection shall be conducted, the extent of test or 
inspection, the criteria for acceptance, the required monitoring intervals, and the 
requirements for recording and reporting data. The URO/MRC Program is the technical 
basis for continued unrestricted operation of the submarine. A submarine can be 
prohibited from submerging if URO/MRC requirements are not met, unless a waiver is 
approved by NAVSEA. 

Audits 


Functional audits to examine policies, procedures, and practices are performed on a 
periodic basis at all activities involved in SUBSAFE work. This includes the 
design/construction yards, shipbuilders, the Supervisor of Shipbuilding (SUPSHIP), 
Naval shipyards, NAVSEA HQ, and the Fleet Type Commander (TYCOM). 
Certification audits are performed on all submarines at the completion of: 1.) new build 
(design/construction yard) and 2.) major maintenance/modemization work (shipyard). 
Audits cover the certification of the design, material, fabrication, and testing. The basis 
for certification is the OQE as discussed above. 


NNBE INTERIM REPORT 


18 



The SUBSAFE Program also includes training and qualification processes and 
requirements. Only select activities are allowed to perform SUBSAFE work, as 
recognized in NAVSEA Note 5000. All of these activities, including the authorized 
design yards, shipbuilders, public shipyards, SUPSHIP, engineering/logistics field 
activities and TYCOM are responsible for developing and implementing necessary 
instructions and procedures to meet the requirements of the SUB SAFE Program. In 
support of these responsibilities a SUBSAFE Program Director (SSPD) is assigned at 
each activity involved in the SUB SAFE Program. 

Key Observations: SUBSAFE Program 

The SUBSAFE concept addresses specific flooding/recovery hazards and places 
special life- cycle emphasis on controlling those hazards. Other hazard control 
boundaries exist on a submarine as shown in figure 2.2. 

There is clarity, uniformity, and consistency of submarine safety requirements and 
responsibilities. Tailoring by program managers is not permitted without NAVSEA 
05 and 92Q approval. 

There is a strong, independent “audit to requirements” assurance organization. 

There is a community- wide (contractor and government) understanding of SUBSAFE 
requirements and a commitment to compliance. 

The SUBSAFE Program and implementing organization (NAVSEA 92Q) are 
relatively immune to budget pressures - no certification, no submarine. 

There is a strong, community- wide (contractor and government), continuing emphasis 
on safety through periodic systematic examination of previous failures and incidents. 
Annual SUBSAFE training is a requirement for the NAVSEA Headquarters 
submarine community. 


3.2 Management and 
Organization 

As shown in figure 3.3, NAVSEA 92 
reports directly to the Commander, 
Naval Sea Systems Command 
(NAVSEA). NAVSEA 92Q is 
responsible to NAVSEA 92 for the 
implementation, administration, and 
coordination of the non- reactor plant 
portions of the SUB SAFE Program, 
and for ensuring compliance with 
SUB SAFE requirements. The 
SUB SAFE Program is guided and 
directed by the Submarine Safety 
Steering Task Group (SSSTG), which 
is chaired by the Deputy Commander 
for Submarines (NAVSEA 92). 



Figure 3.3 Functional Relationship Between Submarine 
Program Managers/Senior Mgmt & SUBSAFE Program 
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Each NAVSEA Submarine Program Manager (PMS) retains responsibility for executing 
the SUBSAPE Program for assigned ships. New- construction submarine program 
managers, such as SEAWOEE (PMS-350) and VIRGINIA (PMS-450), report directly to 
PEG Submarines. As shown in figure 3.3, the PEG Submarines reports to the Assistant 
Secretary of Navy for Research, Development, and Acquisition. The NAVSEA 92Q 
organization reports by way of an independent path through NAVSEA 92 and NAVSEA 
00 to the Chief of Naval Gperations. Eor new construction, NAVSEA 92Q (as well as 
NAVSEA 05 and NAVSEA 04) provides matrixed support directly to PEG Submarines. 
In-service submarine program managers (e.g., PMS-392) report to NAVSEA 92. 
Although they are in the same organization, NAVSEA 92Q maintains its independence 
from the in-service program managers. 

Cognizant NAVSEA teehnical codes provide technical assistance to the PMSs and to 
NAVSEA 92Q as requested to ensure timely resolutbn of problems. The final authority 
for the technical requirements of the SUBSAPE Program resides with NAVSEA 05. 

3.2.1 Technical Authority 

Submarine design, material, and process requirements are contained in the ship 
specification and/or documents cited as applicable references (see section 3.3.2). As 
noted previously, the responsibility for establishing and maintaining technical 
requirements (SUBSAPE fabrication standards, material specifications, etc.) is vested in 
the NAVSEA 05 organization, and in some cases, is formally delegated with a 
specifically defined scope of authority to organizations such as NAVSEA 92T or 
NAVSEA- managed field activities. 

It is important to note that NAVSEA 05 owns the teehnical requirements, separately from 
line program management (new design) and/or organizations implementing 
maintenance/modernization activities. Also, program managers and implementing 
organizations have no authority to "tailor" or waive SUBSAPE or DSS requirements 
without submitting a formal request through the NAVSEA 05 organization. To improve 
efficiency and facilitate NAVSEA downsizing, some authority has been formally 
delegated to implementing organizations (e.g. SUPSHIP Groton or PNS) to disposition 
non-conformances within speeified limits. 

Key Observations: Technical Authority 

The NAVSEA organizational structure provides a capable, centralized, independent, 
technical authority that is responsible for developing and documenting technical 
requirements and standards, and providing requirements clarity and accountability. 
There exists community- wide acceptance that NAVSEA 05 is the teehnieal 
requirements owner. 

Any delegation of technical authority from NAVSEA 05 to implementing 
organizations is clearly documented. 
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The centralized technical authority provides a powerful means to capture, document, 
and use lessons learned to improve future ship designs. 

3.2.2 Independent Compliance Verification 

Compliance with critical SUBSAFE and DSS design and process requirements is 
independently verified by a highly capable, centralized organization, namely NAVSEA 
92Q. NAVSEA 92Q is the owner of the SUBSAEE and DSS processes. It administers 
the programs and monitors for compliance. 

NAVSEA 92Q serves as an independent compliance verification organization. It 
functions to assure that policies, procedures, and practices are in accordance with 
SUBSAEE and DSS requirements, and that they are followed. Compliance is verified 
through the implementation of rigorous functional and certification audit processes. 
Through the requirements stipulated in the SUBSAEE Manual, this organizational 
structure is passed down to each of the SUBSAEE certified activities. The SUBSAEE 
Program Director (SSPD) at each activity is responsible back to SEA92Q to ensure that 
the SUBSAEE Program requirements are established and maintained internally, with 
independent internal quality assurance activities verifying compliance. 

NAVSEA 08 accomplishes its compliance function using a combination of field office 
and headquarters team audits, in addition to each facility performing self-assessments and 
internal audits. 

Key Observations: Independent Compliance Verification 

The NAVSEA management organizational structure provides a highly capable and 
independent safety compliance verification/assessment organization that serves as a 
key management control for SUBSAEE certified activities. 

The compliance verification process is the responsibility of an entity separate from 
program management and the operators (the Elect Type Commanders) of submarines. 
Audit activities draw on expertise distributed across the Navy laboratories, shipyards, 
and other field activities. 

3.2.3 Safety Management Philosophy 

The Navy defines both the “what” and the “how” in the development and implementation 
of safety, in general, and the SUBSAEE Program, in particular. This applies equally to 
both the new design/construction and maintenance/modernization activities described in 
this report. 

Key Observations: Safety Management Philosophy 

— Contractors are not given latitude in meeting SUBSAEE and other critical safety 
system requirements because the leverage gained in the areas of technical excellence 
and risk mitigation achieved through the many generations of submarine 
specifications would be lost with such an approach. 
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3.2.4 Cultural Attributes 


The SUBSAFE Program infuses the submarine Navy with safety requirements 
uniformity, clarity, focus, and accountability. 

The Navy’s safety culture is embedded in the military. Civil Service, and contractor 
community through: 1.) clear, concise, non- negotiable requirements, 2.) multiple, 
structured audits that hold personnel at all levels accountable for safety, and 3.) annual 
training with strong, emotional lessons learned from past failures. Together, these 
processes serve as powerful motivators that maintain the Navy’s safety culture at all 
levels. 

In the submarine Navy, many individuals understand safety on a first-hand and personal 
basis. The Navy has had over a hundred thousand individuals that have been to sea in 
submarines. In fact, many of the submarine designers and senior managers at both the 
contractors and NAVSEA routinely are onboard each submarine during its sea trials. In 
addition, the submarine Navy conducts annual training, revisiting major mishaps and 
lessons learned, including USS THRESHER (SSN-593), Space Shuttle Challenger, and 
USS BONEFISH (SS-582). NAVSEA uses the USS THRESHER loss as the basis for 
annual mandatory training. During training, personnel watch a video on the USS 
THRESHER, listen to a two- minute long audiotape of a submarine’s hull collapsing, and 
are reminded that people were dying as this occurred. These vivid reminders, posters, 
and other observances throughout the submarine community help maintain the safety 
focus, and it continually renews the safety culture. 

The Navy has a traditional military discipline and culture. The NAVSEA organization 
also is oriented to compliance with institutional policy requirements. In the submarine 
Navy there is a uniformity of training, qualification requirements, education, etc., which 
reflects a single mission or product line, i.e., building and operating nuclear powered 
submarines. 

Key Observations: Cultural Attributes 

- Safety is central to the culture of the entire Navy submarine community, including 
builders, designers, maintainers, and operators. 

- In addition to the loss of the USS THRESHER and the USS BONEFISH fire, the 
Navy has made extensive use of the Challenger mishap in its safety training. 

3.2.5 Restructure and Downsizing 

The NASA team met with both Navy and contractor organizations to discuss the impact 
and subsequent innovations implemented in response to the order of magnitude decrease 
in the Navy’s acquisition of submarines in the early 1990s. The following paragraphs 
describe these meetings with NAVSEA Headquarters and Electric Boat. 
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NAVSEA 


Beginning in the early 1990’ s (post Cold War) the Nation experienced a “peace 
dividend” with a subsequent call for reduction in the DoD mission, structure, and 
organization. Consequently, NAVSEA formally initiated a restructuring and downsizing 
of its entire workforce, which inevitably affected the size of the NAVSEA submarine 
community. The overall objective was to reduce the size and structure of the 
organization while assuring that critical defense technologies would be maintained. This 
was accomplished over several years and in a number of discrete steps. 

The first step was a reduction in the number of managerial levels (groups, subgroups, and 
divisions) that existed within NAVSEA Headquarters. The net effect was the flattening of 
the organization from the previous structure that had as many as 1 1 management layers 
between the Secretary of the Navy and the working level engineer. The resulting 
organization provided the required increase in efficiency and flexibility. This initial 
restructuring at NAVSEA Headquarters was followed by a major reorganization in 1992 
of the Navy laboratories that involved transforming the labs from fundamental research 
and development entities to warfare centers. These warfare centers, each with distinct and 
identifiable product lines, now report to NAVSEA. 

The next significant event was a series of DoD base closures, including four Navy 
shipyards, with an attendant loss of significant numbers of engineering personnel 
assigned to NAVSEA. 

Beginning in 1995, and in response to these downsizing events, NAVSEA undertook a 
formally structured approach to assure continued support of critical defense technologies. 
This was accomplished through the development of a warfighting system engineering 
hierarchy that defined the necessary engineering capability requirements. 

By 1998 NAVSEA began to address the aging workforce issue and a serious loss of 
talent. At this time NAVSEA was losing approximately 750 engineers per year through 
retirement and resignations. NAVSEA began by refocusing on its core equities or 
competencies: 

- Setting technical standards and policies, 

- Certifying/validating delivered products, and 

- Providing a vision for the future, i.e., technology infusion/evolution. 

Additionally, the following new engineering categories and career paths were defined: 

- Research and development 

- Science and technology (early stage technology development, i.e., discovery and 
adventure) 

- Subject matter experts 

- Systems engineering 

- Engineering management 
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Most importantly, NAVSEA actively sought and obtained approval to overstaff its 
engineering skill sets with new or recent college graduates in order to provide for a 
logical and systematic transition of its aging workforce. As a result, NAVSEA began an 
active reeruitment program to hire engineering professionals. By 1999 NAVSEA had 
reaehed the breakeven point and by 2002 hired 1000 new engineers with a net gain of 
300. This innovative approach allowed its skilled and experieneed engineers to train and 
mentor the engineering new hires and provide eritieal knowledge transfer while 
sustaining its eore competeneies. 

Key Observations: NAVSEA Restructure & Downsizing 

- NAVSEA eondueted a logical and structured downsizing approaeh that identified 
eore eompeteneies and the engineering workforce requirements neeessary to assure 
eontinued support of eritieal defense technologies. 

- NAVSEA overstaffed its engineering skill sets with reeent eollege graduates to allow 
the skilled work foree to train new hires while maintaining core competencies. 

Eleetrie Boat Corporation 

The NASA team met with the President and senior management staff of Eleetrie Boat to 
discuss their management approaeh to the downturn in submarine produetion from five 
submarines a year to one -half submarines per year that oeeurred in the early 1990s. The 
impaet of that downturn eannot be overstated. In eontrast to NAVSEA, where designers 
and subjeet matter experts eould shift to other projeets and future designs, EB did not 
have this option beeause its sole purpose is submarine produetion. Eor EB the downturn 
beeame a matter of survival, and EB realized that it needed to implement cultural as well 
as management ehanges. 

This management ehallenge was driven by a sense of urgeney to demonstrate to the 
parent eorporation (General Dynamies) that EB eould make money. Working elosely 
with the Navy, EB management and the labor unions came together under this shared 
sense of urgeney. 

Staff reductions were implemented and pink slips were issued based not only on union 
rules governing seniority, but also the need to retain eritieal skills. The EB white-eollar 
staff was also reduced after an internal analysis based on individual skills, experience, 
and training determined each individual's value to the eompany. Thus, senior EB 
management made reduetions not only at the hourly workforee level, but also within eaeh 
layer of management. Throughout the diffieult and painful staff reduction process, EB 
management was careful to ensure that core eompeteneies were retained. 

At the same time the staff was reduced, EB implemented a shift to greater company- wide 
use of digital technology in design, manufacturing, and administrative processes. The 
digital shift was eombined with a proeess re-engineering initiative that streamlined 
eumbersome internal proeesses and aehieved operating effieieneies. 
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Currently, EB is seeking opportunities to broaden its business base by moving to 
submarine maintenance/modernization to supplement the low-rate new 
design/construction environment. 

Key Observations: EB Restructure & Downsizing 

- The Navy worked elosely with EB to ensure that a logieal and struetured approaeh 
was employed to downsize the workforee. There was no "magie pill" for what was a 
painful downsizing process. 

- Changes that were made at EB were driven by the need to survive as a business unit 
within the General Dynamics Corporation. Union and corporate management worked 
together to determine the optimum mix of competencies, seniority, and management 
for the restructure and survival of the company. 

- Having clearly defined, well- documented technical and procedural requirements 
assisted the Navy and EB during the restructuring and downsizing transition. 

3.3 Safety Requirements 

3.3.1 Safety Requirements Management 

As noted previously, the NAVSEA 05 organization is the technical authority responsible 
for all technical requirements including those contained in the SUBSAEE and DSS-SOC 
manuals. 

Within the submarine Navy, design safety requirements are not typically tailored from 
project-to-project or shipyard- to- shipyard. Design safety requirements are not a matter of 
debate with each new submarine or human-rated deep submersible system. Safety 
requirements are virtually outside the trade- space for program/project managers. 

The safety requirements set includes strict life- cycle configuration management 
requirements (re-entry control) and periodic maintenance requirements identified in the 
SUBSAEE Manual as URO/MRCs (Unrestricted Operations/Maintenance Requirements 
Cards). The SUBMEPP organization maintains the URO/MRCs through delegation from 
NAVSEA 05. A more detailed description of the functions and responsibilities of 
SUBMEPP is provided in section 3.4.3. 

It is also important to note that other safety requirements exist, such as those for weapons 
and general hazard categories, as discussed in section 2.2 of this report. 

Key Observations: Safety Requirements Management 

Critical safety requirements and implementation methods are clearly defined. 

Critical safety requirements are protected, and program managers cannot tailor them 
or trade them against other technical or programmatic variables. 
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3.3.2 Submarine Safety Design Criteria 


As noted previously, the ship specification or "ship spec" for each new class of 
submarine is built upon the ship specifications from previous classes of nuclear, deep- 
diving submarines. The ship spec reflects the continuous process of design risk 
identification and mitigation through many successive design, build, and operation 
cycles. The ship spec contains the best that can be achieved given lessons learned from 
previous designs, extensive operational data, and a structured effort on all fronts to 
identify and mitigate hazards and risks in new design components. 

There does not exist a single (human rating) design guidance document that describes 
functional design requirements for system and sub -system redundancy (i.e., fault 
tolerance), structural factors of safety, or crew escape or rescue provisions. While no 
single human rating design criteria or guidance document was identified, the ship spec 
references a suite of documents that govern a new design and which includes as a 
minimum: 1.) the SUBSAFE Manual, 2.) the DSS-SOC Manual, and process standards 
such as 3.) MIL-STD-882D, 4.) the Material Control Standard, and 5.) the Welding 
Standard.'^ 

Key Observations: Submarine Safety Design Criteria 

- There does not exist a single (stand alone) document that proscribes NAVSEA human 
rating design safety criteria or standards. 

- The existing operational design attributes, compiled in the most recent ship 
specification, represent the de facto, evolved human rating design standard. 

3.4 Implementation Processes 

3.4.1 Design Approach 

Evolutionary PesiRn 

Lessons learned from the development and operation of submarine systems over the past 
century have evolved into current processes, practices, and requirements that have served 
to reduce operational risk and uncertainty. 

General Process 


A typical NAVSEA design team includes employees from within NAVSEA 
Headquarters, experts resident at the Naval laboratories, and experienced design yard 
contractors. The NAVSEA ship’s design manager and most of the team are NAVSEA 05 
employees. Memoranda of Agreement (MOA) are typically used when laboratory 
workers and other organizations are required to participate on the design team. The 


NAVSEA 0948-LP-045-7010, Material Control Standard (Non-Nuclear) 

NAVSEA T9074-AD-GIB-010/1688, Requirements for Fabrication, Welding and Inspection of 
Submarine Structure. 
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ship’s design manager leads the technical team and employs Major Area Teams (MATs) 
consisting of multiple Systems Integration Teams (SITs). 


This organizational structure is mirrored in the design yard, creating a collaboration of 
customer and contractor, with NAVSEA maintaining ultimate technical authority. Each 
SIT may have one or more integrated product team (IPT). Individual IPTs in turn call 
upon the independent technical analysis and evaluation support of experts located at 
various Navy laboratories. Systems engineering takes place within the MATs, SITs and 
IPTs. In effect, the ship’s design manager serves as the systems engineer. The Change 
Control Board represents the key system engineering management forum. 

VIRGINIA Class Design Evolution 

The original plan was to build 30 SEAWOEE submarines as a follow-on to the 688-EOS 
ANGEEES Class attack submarines, but only three were actually funded and built. The 
end of the Cold War contributed to the decision to limit production. Both NAVSEA and 
EB realized that a change had to be made in how submarines are designed and produced. 
The VIRGINIA Class submarine is now intended as the follow-on ship design to the 
SEAWOEE Class and will augment and replace retiring 688s. The review for the 
VIRGINIA Class included the design/build concept and affordability at low production 
rates. 

Electric Boat, contracted as the VIRGINIA Design Yard, developed design/build teams 
including experts from its design community and NAVSEA technical expertise to ensure 
the design was technically optimized. Experts from the shipyard were employed to 
ensure fabrication/construction efficiencies were captured in the design. Design/build 
represents a full commitment to concurrent engineering principles and practices, 
involving all of the stakeholders in the design, including designers, fabricators, suppliers, 
operators, maintainers, and approval authorities. 

As part of the effort to achieve an affordable design, NAVSEA engaged its contractor 
team to examine requirements in search of ways to reduce cost while maintaining safety 
requirements. As the program progressed towards construction, development of a 
contractor construction partnership was established in which Electric Boat and Newport 
News Shipbuilding cooperated instead of competing for the contract. Newport News 
Shipbuilding was brought to the design/build team to further ensure that fabrication 
efficiencies supported its construction processes. 

The partnership enabled: 

1) Early collaboration between the government buyer and the design/build team. 

2) Early collaboration in the concept development phase to assure “design for 
manufacturability /maintainability,” and 

3) Examination of safety requirements documents to seek ways to reduce costs while 
maintaining the intent and force of the requirements (e.g., approval of equivalent 
contractor processes or procedures). 
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Key Observations: Design Approach 


- New ship designs are evolutionary. Lessons are learned and applied from one 
submarine class to the next. 

- The VIRGINIA contract partnership agreement (EB and Newport News 
Shipbuilding) enabled a strong degree of collaboration. 

- The newest design class, VIRGINIA, employs design for manufacturability (six- 
sigma) concepts. NAVSEA and EB consider the approach, referred to as 
design/build, as critical to achieving affordability. 

- Early and continual collaboration between government and contractors reduces 
programmatic risk by emphasizing and strengthening design for manufacturability. 

3.4.2 Processes/Tools 

EB Quantitative Methods 

Quantitative safety assessments are essentially deterministic in the Navy submarine 
program and driven by the following three documents: 1.) the SUBSAEE Manual, 2.) the 
DSS-SOC Manual, and 3.) DOD MIE-STD-882. 

A traditional hazard/impact evaluation process is used for safety assessment. It consists 
of the following steps: 1.) hazard analyses, 2.) identification of hazards and 
recommendation(s) for resolution or acceptance, 3.) resolutions or acceptance of hazards 
and 4.) hazard/impact closure. Resolutions may involve: a.) design modifications to 
resolve hazard/impact, b.) tests to verify hazard impact on risk acceptability, and c.) 
actions/procedures and training to resolve issue/hazard. 

Reliability predictions are performed separately for humans and hardware. 

Human reliability assessments are based on Swain’s THERP (Technique for Human 
Error Rate Prediction). Hardware reliability assessments rely on electronic parts 
prediction using PRISM, a computer tool developed by the Reliability Analysis Center. 

Mission reliability/operational availability is performed with the following computer 
tools: TIGER (NAVSEA Monte Carlo program), and RAPTOR (ARINC Monte Carlo 
program). Other reliability analyses used include PMECA (failure modes and effects 
criticality analysis), fault tree analysis using Eault Tree Plus (ItemSoft), COTS 
(commercial off-the-shelf) obsolescence & sparing analysis, and Reliability Calculator 
developed by Electric Boat. Hardware maintainability analysis is based on DOD’s Mi l - 
HDBK-470. 
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Key Observations: EB Quantitative Methods 


Traditional safety /hazard assessments are used and rely extensively on the vast 
historic data (design/build/test/maintenance and operational experience) of the US 
Navy Submarine Program. 

Reliability and maintainability assessments are performed according to traditional 
DOD methods and also rely heavily on the vast historic data of the program. 
Probabilistic Risk Assessment does not appear to be used at any significant level. 

Risk assessment is applied qualitatively/categorically using a DOD-prescribed 5x5 
likelihood versus consequence matrix that is similar to the one used by NASA. 

NAVSEA HQ Role in Software Development and Certification 

The ship control system on SEAWOLF is a limited “fly-by- wire” design (analogous to an 
aircraft flight control system). The steering and diving system is driven by computers, 
but it can be decoupled and revert back to a traditional system completely controlled by 
hydraulics. The USS VIRGINIA is the first submarine that will be primarily fly-by- wire. 
With the advent of the VIRGINIA program, NAVSEA was asked to develop a software 
verification and assurance plan for the ship control system. To assist in this endeavor, a 
software safety certification process action team or PAT was also established including 
NAVSEA 05U, NAVSEA 92Q, and the Program Manager. The challenge was to 
identify the basis for certifying ship control software systems. The team looked at how 
software interacted with systems within the SUBSAFE and recovery system boundaries, 
in particular steering and diving. Ten rules were developed pertaining to control of 
critical submarine functionality (e.g., flood control, maneuverability, emergency blow 
control, etc.). An overarching rule was to assure that software could not wrest control of 
the submarine from the crew. In a related effort, the NAVSEA Chief Engineer (SEA05) 
and PEG SUB jointly chartered the Safety of Flight Industry Survey Team to conduct a 
benchmarking review of aviation industry best practices to assist in developing the 
NAVSEA approach. The current NAVSEA process does not yet reflect the results from 
the Safety of Flight industry survey because the results of this survey are still being 
documented and reported within NAVSEA. 

Multiple boards and control processes were established at the NAVSEA level to address 
various aspects of the software design challenge, including: 1.) the VIRGINIA design 
team software safety IPT, 2.) the safety certification PAT, 3.) the Safety of Flight Team, 
4.) the WSESRB software safety review panel, and 5.) the submarine software safety 
technical review panel (SSS-TRP). 

Key Observations: NAVSEA HQ Role in Software Development 
and Certification 

- The VIRGINIA Class represents the first NAVSEA application of fully fly-by- wire 
technology for ship control system design; accordingly, its software management and 
control processes are evolving. 
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- The current approach represents a comprehensive and conservative management 
approach to assuring software capability and fidelity. 

EB Software Development and Implementation Assurance 

The Electric Boat software engineering, testing, and software quality assurance functions 
are all responsible to one vice president with several directors. The safety function is 
responsible to a different vice president. The software engineering department is 
matrixes personnel to various departments to cover these functions as needed. The 
benefit of having all software personnel managed by a central manager is the ability to 
manage the resources, balancing and trading off where needed. 

While there are many advantages to having a centralized core software group, drawbacks 
include: 1.) a single string organizational approach toward implementing software 
assurance (including software safety) that is vulnerable to changes in personnel, and 2.) 
separate reporting to management from matrixed software assurance and safety functions 
that can lead to confusion. However, each director still has the responsibility for 
providing OQE that software development is implemented properly. 

There is heavy use of EB personnel at the contractor/vendor sites with inspections at the 
source. This eliminates waiting for delivery, discovery of problems, and the return of 
unsatisfactory items. Requirements are very detailed and include safety critical criteria. 
Procurement is trained to ask for (and expects) high quality Hazard Analyses even for 
software. Suppliers are checked for the health and viability, they are certified, and their 
core knowledge examined and checked for maintainability. All this leads to a long-term 
relationship with known vendors, and each works to support the other. 

The EB software development processes, including software assurance, can be described 
as follows. Ship specs come in with little to no software specifications. A ship spec is 
broken down into component specs with software identified to the component level. The 
EB software team then creates performance-based (vs. functional-based) specifications 
for the software. The request for proposal (REP) has standard software clauses, which 
detail how the software is to be developed (including quality assurance, configuration 
management, and verification & validation). EB provides a template for the expected 
vendor software development plan. Separate software costs are requested, but a 
software- specific work breakdown structure (WBS) is not always supplied for software 
integral to a major system component. 

The vendors are challenged to demonstrate capability in software even when it is not a 
primary deliverable. Subcontractor management must show the roll down of EB’s 
software qualification requirements. This tends to increase subcontractor management 
costs but pays off in the long run. If a sole or single source is to be selected, the REP is 
skipped and the software spec and WBS are written into the purchase order. The REP 
evaluation has both technical and cost evaluation criteria that are project dependent. The 
contract is let with EB software specs and WBS built in. Also, included is a list of 
expected deliverables including metrics (with specifics on which ones and how and when 
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to measure), products, test reports, and approval needs (requirements, design, test 
procedures and results, estimates for regression testing, etc.). Vendors are also asked for 
a Preliminary Hazard Analysis. 

A software statement of work (SOW) is created for each componeit, and the software 
development plan is examined to assure it contains all that the template provided to the 
contractors/vendors required. This software development plan is levied on the vendors 
and must include an organizational layout, configuration management, quality assurance, 
test plans, system safety plans, and, if required, software safety plans. The EB software 
team, like its hardware counterpart, is stationed with the vendors and works with them on 
a day-to-day basis. Weekly software status telecommunications and video 
teleconferences are held to keep EB software management informed and up to date. Peer 
reviews are held for all software products. Safety and assurance are involved with all 
peer reviews, both in-house and at the vendors. 

Safety analyses consist of safety scenarios, fault tree analysis (ETA), and selected failure 
modes and effects analyses (EMEAs). Hazards are first postulated from historical 
experience and data, corrective maintenance records, and operational experience. The 
TIGER^^ tool is used for modeling operational availability and failure rates where there 
are few historical data available. 

Analyses are performed on software requirements and design, as well as any changes to 
the requirements or design. Eor the VIRGINIA Class submarine ship control system, EB 
also has Navy Independent Verification and Validation (IV&V). The Navy Program 
Office (PMS450) provides an independent check of the critical software through 
assignments to its Naval laboratories. 

Because of the team-oriented Integrated Product and Process Development (IPPD) 
process used for VIRGINIA Class development, this IV&V effort is conducted within the 
scope of the Ship Control System IPT rather than reporting separately to a higher level of 
management. Certification begins once code and unit testing are complete, and 
integration with the hardware allows for requirements verification. Systems certification 
and environmental quality assurance checks are also performed. 

Key Observations: EB Software Development and Implementation Assurance 4mm 

EB implements a very rigorous software contracting process that assures that safety 
requirements are appropriately Bowed-down and met at each level (i.e.. Navy to EB 
to subcontractors). 

EB Digital Design and Engineering Environment 


EB has worked hard to improve its communication and data maintenance. To help 
maintain expertise, much of it is built into the large suite of tools it has assembled for 
design, analysis, visualization, testing, and simulation. EB’s shift away from traditional 

TIGER is a reliability /availability modeling tool developed and used by NAVSEA 
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draftsmanship, and its heavy reliance on 3-D design suites, has helped maintain and 
communicate its work within a distributed design and manufacturing environment. EB 
also uses video telecommunications in conjunction with its 3-D simulations on a regular 
basis to conduct design discussions and work interactively as a distributed team. 

Key Observations: EB Digital Design and Engineering Environment 

EB use of software-based design tools (e.g., CATIA) has helped them maintain and 
communicate its work within a distributed engineering design, manufacturing, and 
maintenance environment. 

Human Eactors in the Design Process 

The Navy considers the human/system interfaces for both operators and maintainers as an 
integral part of the submarine design. Requirements have evolved over successive 
generations of submarine design. The SUBSAEE and DSS-SOC manuals provide human 
factors engineering requirements related to critical SUBSAEE systems maintenance tasks 
and related tasks for operation of equipment during emergency conditions. The 
formalization of a dedicated human interface analysis approach in overall design is a 
relatively new endeavor. 

In October 2002, a new organization, NAVSEA 03 Human Systems Integration in 
System Design, was created within NAVSEA. The NAVSEA 03 mission is to optimize 
total system performance and life cycle cost by ensuring that the operator is engineered 
into the system, from the beginning of the ship design process through upgrades and 
enhancements. NAVSEA 03 will establish corporate NAVSEA policy, technical 
standards, human performance metrics, and ensure usability of Navy systems. The Navy 
has made the investment in NAVSEA 03 to shape the Navy’s future by institutionalizing 
optimal staffing, transitioning technology, and advocating human/system interface as an 
essential element of total ship engineering. 

Currently, the Navy is consciously engineering maintenance and operations activities into 
its newest design, the VIRGINIA Class. EB is using computer aided design tools to 
determine how maintenance and operations will be performed prior to the fabrication of 
system designs. This allows the Navy to consider the human-system interfaces in early 
concept design, make decisions and trades that maintain safety, optimize efficiency, 
optimize crew size, and minimize cost throughout the system life cycle. 

Key Observations: Human Factors 

NAVSEA systematically considers life-cycle human/machine interface requirements 
in the design of submarines. 

NAVSEA 03 - “Human Systems Integration in System Design” was created to ensure 
that human/system interface is an essential element in total ship systems engineering. 
Human Eactors Engineering requirements, unique to SUBSAEE and deep submergence 
systems, are included in the SUBSAEE and DSS-SOC manuals. 
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3.4.3 Work Planning and Authorization Processes (Maintenance/Modernization) 


The Navy plans its maintenance/modernization tasks well in advance of task completion. 
The Navy’s “fast start strategy” requires that one hundred percent of the first sixty days 
of task instructions be written, reviewed, and authorized prior to the submarine reaching 
an industrial facility (e.g., dry dock). Additionally, planning for sixty percent of all work 
that will be done during unavailability (e.g., non- operational maintenance period, major 
overhaul, etc.) must be completed prior to the submarine reaching the industrial facility. 
The advance planning process and the work instruction review process provide layers of 
defense that allow for the early detection and correction of potential errors, hazards, and 
safety problems before the ship reaches the facility, before maintenance/modemization 
tasks are initiated, or before an accident occurs. 

The Navy’s fast start strategy also allows them to prepare long-range plans and forecast 
the need for materials, parts, and personnel resources. By specifically identifying and 
writing task instructions far in advance, and by keeping detailed information about each 
component unit (procedures required, number of hours to install, and certification 
required), the Navy knows exactly how many personnel are needed and their training 
requirements. Using this information and data on workforce reductions and attrition, the 
Navy can project the training needed to maintain the minimal core of certified 
professionals to safely perform maintenance and modernization tasks. 

SUBMEPP 


Navy Submarine Maintenance Engineering, Planning and Procurement (SUBMEPP) is 
responsible for planning what work will be done during maintenance overhauls and when 
it will be done. SUBMEPP develops and maintains maintenance standards and test 
procedures (with integrated SUBSAEE, DSS-SOC and URO/MRC requirements) that 
support class maintenance plans with further definition of the minimum technical 
requirements that have to be achieved for a particular maintenance action. Maintenance 
standards are provided to the Ship Availability Planning, and Engineering Center 
(SHAPEC) as definition of what work in the class maintenance plan is to be conducted. 
SUBMEPP maintains the class maintenance plan that is used as a basis for the 
development of an availability work package that is provided to SHAPEC. SHAPEC 
then provides technical work instructions to Portsmouth Naval Shipyard (PNS), and all 
other shipyards where depot level submarine maintenance is performed, with details on 
how the work specified in the class maintenance plan is to be conducted. The SHAPEC 
process is discussed in detail in the next section. 

Although collocated at PNS, SUBMEPP is a separate entity that reports to NAVSEA 92. 
SUBMEPP is jointly funded by NAVSEA 92 and the Elect Type Commanders 
(TYCOMs). NAVSEA 05 has a MOA with SUBMEPP (through NAVSEA 92T) to 
delegate technical authority for certain issues to SUBMEPP. Any issues outside of 
SUBMEPP’ s defined technical authority are referred back to NAVSEA for resolution. 
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SUBMEPP maintains the Joint Fleet Maintenance Manual (JFMM) for the TYCOMs. 
The JFMM document supports performance of maintenance carried out by the Fleet 
(outside of shipyards). SUBMEPP also supports the TYCOMs by identifying 
maintenance work that can be scaled back or deferred to the next availability to meet 
budget or schedule constraints while maintaining mission capability and safety 
requirements. 

SUBMEPP heavily employs Reliability Centered Maintenance (RCM) to achieve cost 
savings by leveraging multi-class reliability data developed over the years that supersede 
ultraconservative maintenance plan requirements originally developed in the late 1960’s. 
RCM is conducted at the local component, system, and ship level to identify “applicable 
and effective maintenance tasks that are necessary to achieve required reliability of 
systems or equipment.”^ ^ 

SUBMEPP also employs other standard tools and techniques in maintenance planning 
such as FMEA and root cause analysis, but only at the component level. FMEA does 
include human factors as a failure mode where it is considered a credible failure. 

NAVSEA 92Q conducts functional audits of SUBMEPP to validate its processes and 
maintenance standards. In essence, all audits of Naval shipyards are implicitly an audit 
of SUBMEPP, as its products are used by all. SUBMEPP is involved in sustaining 
certification, but it is not directly involved in the certification process. 

In summary, the SUBMEPP organization has the following overall roles and 
responsibilities: 

- Establish and maintain Class Maintenance Plans, 

- Maintain baseline and ship -specific work packages (depot level), 

- Maintain inventories, schedules, and refit work packages (intermediate 
maintenance activity level), 

- Develop maintenance standards, 

- Administer the URO/MRC program, 

- Employ Reliability-Centered Maintenance (RCM) principles and practices, 

- Implement Condition- Based Maintenance (CBM), 

- Establish testing procedures and maintain test documents, 

- Provide material procurement support, and 

- Provide maintenance data collection/review/trend analysis and feedback 
mechanisms for continuous improvement. 

Key Observations: SUBMEPP 

SUBMEPP advanced planning is a key element in ensuring submarine safety. This 
approach provides management, engineers, and technicians ample time to evaluate 


SUBMEPP 9081-083-0589, “SUBMEPP Reliability Centered Maintenance Handbook.” 
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task steps and detect/correct procedural errors, thereby increasing the likelihood that 
activities will be performed safely. This approach also ensures that needed parts, 
tools, and personnel are available when the task will be performed. 

SUBMEPP is the “detail” organization (e.g., checking tolerances and factors of 
safety) assuring that the right maintenance work is done when it needs to be done and 
in the right way. 

SUBMEPP advanced planning also provides valuable data that allows the Navy to 
make informed decisions concerning workforce staffing, training, and retention. 

Ship Availability Planning and Engineering Center (SHAPEC) 

NAVSEA established SHAPEC by ship class (surface ships, carriers, and submarines) to: 

Streamline the process for determining technical, planning, and material requirements 
for ship work. 

Standardize planning products, practices, and procedures. 

Ensure sharing and reuse of planning products by both government and private sector 
customers for depot level ship work and development of appropriate metrics, and 
Provide libraries of reusable planning products for use by all ship maintenance 
activities. 

Portsmouth Naval Shipyard is the single activity responsible for implementation of the 
SSN 688 Class SHAPEC process for development and maintenance of class planning 
products. This includes centralized development of selected, reusable maintenance, 
modernization, and repair planning products for SSN 688 Class availabilities at 
geographically dispersed sites. SHAPEC establishes product development guidelines, 
maintains a Planning Product Eibrary, and when workload demands, subcontracts to 
other activities (usually other public shipyards) to develop SSN 688 Class advance 
planning products. The SHAPEC process has significantly reduced the cost of availability 
planning through centralized development and reuse of planning products. If a large 
number of VIRGINIA Class submarines are constructed, SHAPEC may expand to 
provide work planning for this class as well. 

PNS has also developed the Baseline Project Management Plan (BPMP) which represents 
the advanced planning process applied to all types of major availabilities including Depot 
Modernization Periods (DMPs), Engineered Overhauls (EOHs), and Engineered 
Refueling Overhauls (EROs). The BPMP assigns a project team, led by a project 
superintendent, for each submarine availability. This project superintendent is the single 
individual responsible for all maintenance activities performed on a submarine, including 
all SUBSAEE work, during its time in the shipyard. 
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SHAPEC uses NAVSEA’s primary technical requirements and directives (e.g., 
DDGOS,'^ the SUBSAEE Manual, etc.) and URO/MRCs in its centralized planning to: 

- Ensure consistency of work instructions and planning products, 

- Ensure best practices, lessons learned, and the most efficient production processes 
are available to all activities involved, 

- Integrate corporate feedback into advance planning products, and 

- Emphasize reusability of non-nuclear planning products for availabilities in 
multiple regions for similar work, eliminating duplication of planning. 

Performance of the SHAPEC function requires PNS to interface with Elect Commanders, 
SSN 688 Class Planning Yard (design agent), SUBMEPP, other shipyards, and the 
NAVSEA program manager in areas of funding, specifications and overall process 
requirements. 

The SHAPEC process requires shipyard technical support organizations to provide: 

- Engineering and planning support to implement and maintain SHAPEC products, 

- Planning products for new and emergent work during an availability, 

- Test procedures and test planning support, 

- Networked production schedules, 

- Technical problem resolution (i.e. Trouble Desk), 

- Material management/procurement, 

- Resolutions for departures from specifications, 

- Certifications, and 

- Planning support for nuclear/non- nuclear interface work. 

Key Observations: SHAPEC 

- SHAPEC was created to place the detailed planning for maintenance/modemization 
availabilities for the SSN 688 Class of ship under one entity. This process has 
reduced cost, complexity, and redundancy of efforts, and it has made available the 
best practices, lessons learned, and most efficient production processes available to 
all public shipyards performing maintenance activity. 

3.4.4 Upgrade/Modernization: NAVSEA Fleet Modernization Program 

The Navy’s Fleet Modernization Program (FMP) closely parallels the Shuttle Upgrades 
Program. The goals of the FMP are to: 

- Improve safety, reliability, maintainability, habitability, and environmental 
compliance of ships and equipment, 

- Improve ships’ capabilities and material condition by installation of approved 
alterations/ modifications, and 


Deep Diving SSBN/SSN Submarines (DDGOS) General Overhaul Specifications 
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- Increase fleet readiness by improving standardization within ship classes. 

The FMP is documented in the Fleet Modernization Management and Operations 
Manual. This manual defines the policy, processes and procedures for accomplishing 
all changes, modifications, and alterations to ships and equipment in the fleet. Coupled 
to the FMP is an OPNAVINST (Chief of Naval Operations Instruction) that provides the 
categorization and criteria for ranking candidate ship alterations (SHIPALTs). 

A SHIP ALT is generated when any change in hull, machinery, equipment, or fittings 
involves a change in the design, material, number, location, or relationship of component 
parts of an assembly. The SHIP ALT process maintains configuration management for 
any change or modification during concept proposal, application, approval, and funding 
phases through final accomplishment. SHIPALTS can be for something as simple as a 
change in a bolt grade, or as complex as the addition of a new weapon system. The 
process is basically the same for the full range. 

Key Observations: NAVSEA Fleet Modernization Program 

NAVSEA manages configuration of modernization efforts via use of a well- 
documented SHIP ALT process. 

Priority is placed on safety, reliability, maintainability, habitability, and 
environmental compliance. 

The FMP is oriented toward long-term upgrade planning (5-10 year horizon). 

3.5 Compliance Verification Processes 

The certification processes for both new submarine construction and certification after a 
maintenance/modernization activity are rigorous and contain multiple checks and control 
processes to assure that work has been performed in accordance with all requirements, 
and in particular, the SUBSAFE and DSS-SOC requirements. The scope of compliance 
verification includes work control and review processes including quality assurance, 
inspection, and surveillance. In both cases OQE forms the paper trail or documented 
basis to support certification decision makers. 

3.5.1 Electric Boat Quality 

The new design/construction OQE acquisition process was examined for the commercial 
shipyard (Electric Boat) case. Principal government participants include the Supervisor 
of Shipbuilding (SUPSHIP), the NAVSEA 92Q SUBSAEE organization, the NAVSEA 
08 Naval Reactors Program, and the submarine program manager. 

EB performs the work and accordingly plays a central role in the certification process. 
Key EB organizations include the SUBSAEE Director, Supplier Quality, and Quality 
Assurance (inspection). Others include the Quality Certification group that maintains 


Document # SL720-AA-MAN-010/020, “Fleet Modernization Management and Operations Manual.” 
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OQE records, and the Material Management organization that oversees material control. 
Supplier Quality conducts intensive audits of EB’s critical suppliers. Quality Assurance 
conducts in-process inspections of work conducted at EB. 


SUPSHIP Groton performs ongoing oversight of EB contract performance including 
SUBSAEE functional audits, SUBSAEE certification audits for each submarine as it 
approaches completion, inspection/auditing of work performed, and procedure reviews. 
SUPSHIP also assigns delegations (subcontracts) to DCMA to perform inspections at 
selected suppliers and for selected commodities. 

NAVSEA 08 has a resident office at EB that performs ongoing oversight of work 
performed at EB. The NAVSEA 08 Naval Reactor Program also conducts functional 
audits of critical suppliers. NAVSEA 08 also assigns delegations to DCMA to perform 
quality assurance surveillance and inspections at critical suppliers. It should be 
emphasized that the NAVSEA 08 quality program operates independent of and in parallel 
with the non-nuclear quality program activities. Additional information on NAVSEA 08 
is provided in section 3.7. 

Key Observations: Electric Boat Quality 

- SUPSHIP serves as a strong, independent, government quality assurance oversight 
organization located at the contractor’s facility. 

- NAVSEA 08 represents a second strong, independent, government quality assurance 
oversight organization also located at the contractor's facility. 

- Multiple assurance processes (EB, SUPSHIP, DCMA, and NAVSEA) extend down 
into the EB supply chain. 

3.5.2 Portsmouth Naval Shipyard (PNS) Quality 

PNS implements a rigorous quality management process. Eor non-nuclear work, the 
process is based on an extension of basic quality management process contained in ISO 
9001 (1994). Eor nuclear work, more rigorous controls are specified, as appropriate, 
based on the complexity and critical nature of the task being performed. 

Requirements for quality assurance are contained in NAVSEA TE855-AA-STD-010, 
Naval Shipyard Quality Program Manual (QPM) implemented at PNS by Technical 
Specification 0942-899-017E. Requirements for the critique and trouble report process at 
PNS are detailed in Chapter 1.10 of NAVSHIPYD INSTRUCTION 4730.14, Work and 
Test Manual (WTM). These requirements come from the parent documents NAVSEA 
4700.17, Preparation and Review of Trouble Reports, NAVSEA 0905-485-6010, Manual 
for the Control of Testing and Ship Conditions, and the QPM. 

The PNS QPM addresses production quality assurance (QA), engineering QA, critique 
and trouble report processes, audit and surveillance programs, inspections, and material 
receipt inspection. PNS conducts metrics-based quality performance measurements using 
a database-driven. Quality Performance System that was developed corporately by all the 
Naval shipyards. 
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The non-nuclear quality function at PNS is distributed across multiple organizations. 
These groups come together in the SUBSAFE Improvement Committee (SSIC) and the 
Quality Council. The Quality Council/SSIC is a team comprised of the Engineering 
Quality Division (Code 200Q), the SUBSAEE Director (Code 200S), the Non-nuclear 
Inspection Division (Code 133), the QA Engineering and Analysis Division (Code 136), 
the Test Engineering and Planning Division (Code 246), and the Production Resources 
Support Office (Code 904). The SSIC and the Quality Council meet twice a month. 
Significant quality issues are formally presented to the Shipyard Commander (Code 100) 
and senior shipyard management monthly in a Quality Performance Meeting. 

A key feature of the PNS quality program is the critique process, a management tool 
designed to address and correct quality problems within the shipyard. PNS follows 
NAVSEA guidance and conducts approximately 140 non-nuclear critiques per year. The 
critique process is used to evaluate, document, and disposition material, process 
engineering, and work execution problems. The process results in issuing a critique 
report, and some events that are critiqued are also documented in trouble reports. 

Trouble reports are associated with significant problems involving personnel injury, 
equipment damage, or those that provide a significant lesson learned for other shipyards. 
Trouble reporting is standardized across all Navy shipyards to document and disseminate 
the most significant problems as lessons learned. Critique and Trouble Report meetings 
are conducted within a day or two of problem occurrence. Trouble Reports (TRs) are 
then transmitted to and shared with other Naval shipyards. 

Important to the success of each critique is an initial investigation into the facts of the 
event. The meetings begin with independent statements provided by the personnel 
involved in the problem. The emphasis of the process is to quickly identify all of the 
relevant facts and consequently the problems associated with the event so that subsequent 
casual analysis yields appropriate and effective corrective actions. 

Important to the success of each critique is an initial investigation into the matter. The 
meetings kick off with statements provided by and questioning of the personnel involved 
in the problem. The local union agreement subjects union members to the critique 
process; however, the results of the critique process cannot be used in disciplinary actions 
against a worker. 

Determining why the mistake occurred is the most difficult part of the critique process. 
Critique chairpersons are trained in root cause analysis, but despite investigating various 
root cause analysis tools, PNS has not found a clear winner. PNS generally arrives at the 
answers by iteratively asking “Why?” the problem occurred in order to drill down to the 
root cause (the “Eive Whys"). Root cause is also revisited during review and issue of the 
reports, done in a weekly meeting of a Critique and Trouble Report Review Board, 
consisting of PNS organizational codes 136, 200S, 200Q, 904 and 246. 

Notification of a problem that could have an impact at another shipyard is required within 
24 hours by submission of a Preliminary TR. Issuance of a final TR, including root cause 
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analysis, determination of problem scope, and identification of preventive measures is 
required within 10 days. For the most severe problems, PNS convenes an Incident 
Review Board (IRB) comprised of the regular critique panel members plus all senior 
management. The IRB process involves development of a problem-cause corrective 
action matrix. IRB TRs require sign-off by all levels of PNS management before being 
sent out to other shipyards. 

The Quality Performance System (QPS) measures quality performance at PNS and other 
shipyards. Each shipyard’s information is available to the others. The QPS database 
specifically tracks personnel and process deficiencies. It does not document equipment 
failures, engineering analysis, or lifting and handling problems. The QP Index (QPI) is 
an overall quality index that is designed to track quality performance compared to an 
expected or normal quality goal. QPS is populated by entering all deficiency 
documentation over the entire range of severity, for example, critique and trouble reports, 
deficiency reports, and surveillance reports. QPS data is regularly analyzed to identify 
emerging trends or problem areas that should be addressed proactively. While QPS is 
populated by a single organization at some other shipyards, PNS employs a decentralized 
approach with part-time responsibility assigned to several organizations across the 
shipyard. 

PNS runs each availability as a smaller, dedicated shipyard using the Project 
Management concept. PNS shop personnel are directly supervised by the project Zone 
Manager when assigned to a project. The PNS shops become invested in the ship, and a 
sense of pride develops that fosters quality throughout the maintenance activity. 

PNS quality also performs material receipt inspection/certification services for the Level 
1^'^/SUBSAFE Stock Program, which supplies material to all Level 1 and SUBSAFE 
material installation activities. PNS quality conducts audits of selected processes 
(mandated by the QPM) that normally are not reviewed in a SUBSAFE certification 
audit, such as metrology and calibration. PNS Procurement Quality Control (PQC) 
processes ensure the adequacy and certification of all Level 1 material. The Naval 
Inventory Control Program orders material and components for maintenance with DCMA 
involvement. PNS performs a material receipt inspection program that includes non- 
destructive test and evaluation at three levels: 1.) generic, 2.) semi- quantitative analysis, 
and 3.) full quantitative analysis of Level 1 critical materials and materials used within 
the SUBSAFE boundary. 

The Re-entry Control (RFC) certification process involves task group instructions (TGI) 
for work and quality assurance testing to be done within the SUB SAFE boundary. REC 
paperwork tracks work and testing, and final certification requires the signature of the 
PNS QA Office. 


Level 1 - any material or equipment critical to safety and/or mission. 
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Key Observations: PNS Quality 


- PNS quality processes are integrated into maintenanee availability activities in order 
to achieve maximum assurance of complianee. 

- Diseipline and individual worker integrity are considered critical to the quality 
assurance process. 

- Honest mistakes are not punished. Not reporting mistakes can subject a worker to 
discipline because the Navy emphasizes full disclosure in the reporting process in 
order to be able to promptly identify and fix problems. Workers are made aware of 
the consequences for failure to report. 

- The Projeet Management approach provides workers with a sense of continuity and 
pride of ownership for the work to be performed on a given submarine. 

- The Navy uses a standardized trouble reporting system across all shipyard/repair 
facilities. This standardized process helps facilitate the sharing of significant 
problems with other shipyards. 

3.5.3 Naval Sea Logistics Center Detachment Portsmouth 

The Naval Sea Logistics Center (NAVSEALOGCEN) Detachment Portsmouth is a field 
aetivity of the NAVSEA Eogistics Directorate (SEA 04) and serves as the Naval Sea 
Technical Agent for developing, maintaining, and assessing life-cyele logistics support 
policies, proeedures, and data systems. It was created to be the interfaee between 
engineering and logistics, and it performs a wide range of logistic support functions 
working closely with the Navy and other government organizations to correet systemic 
problems and to identify design procedural enhancements. NAVSEAEOGCEN is 
located in Portsmouth, NH, and is a fully operational organization under a Direetor who 
reports to the Commanding Officer of the NAVSEA Eogistics Center in Mechanicsburg, 
PA. This detachment develops and administers the supplier past performance automated 
information systems described below. 

Product Data Reporting and Evaluation Program (PDREP) 

The PDREP is an automated information system designed to track quality and delivery 
performance on material/services procured by the Navy. Data is collected from all Naval 
Systems Commands on a daily basis and is maintained in the following records on the 
database: Contractor CAGE Information, Debarment/Suspension, Contract Delivery 
Data, DEA Contractor Alert Eist, GIDEP Alerts, Material Inspection Records, Product 
Quality Deficiency Reports, Qualified Product Eist, Special Quality Data, Surveys, and 
Test Reports. The application offers a wide selection of standard, management, and 
graphical reports. Also, a powerful ad-hoc feature allows users to design their own 
reports. 

Eeedback is a means of providing PDREP users the opportunity to request modifications 
to reports or to make suggestions that would enhance any NAVSEAEOGCEN 
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application/program. The Feedback site assigns the appropriate knowledge expert for 
completion/resolution . 

Level 1/SUBSAFE Verification 

The Level 1/SUBSAFE verification system is a web enabled information system for 
verifying Level 1/SUBSAFE material markings at the end user level (e.g., shipyards, 
fleet, etc.). This verification is a last check to assure material has been certified as Eevel 
1/ SUBSAEE prior to being installed on a ship. The Eevel 1/SUBSAEE verification 
program allows end-users to access the Product Data Reporting and Evaluation Program 
(PDREP) database to retrieve the information they need when they need it. This 
innovative approach provides round-the-clock coverage and has increased customer 
satisfaction. 

Supplier Audit Program 

The Supplier Audit Program is focused primarily on process control. It is not intended to 
take the place of pre-award surveys, product- oriented surveys, or quality system reviews. 
Participation in the Supplier Audit Program is open to NAVSEA, Naval Inventory 
Control Point, and public and private shipyards that have a responsibility to monitor the 
procurement of critical, controlled material. Primary participants include NAVSEA, 
Electric Boat Corporation, Newport News Shipbuilding, Naval Inventory Control Point, 
Portsmouth Naval Shipyard, Norfolk Naval Shipyard, Puget Sound Naval Shipyard, Pearl 
Harbor Naval Shipyard, and DCMA. 

Red/Yellow/Green (RYG) Program 

The RYG Program is a Navy/ Air Eorce automated tool designed to help reduce the risk 
of the government receiving nonconforming products and late shipments. RYG classifies 
the risk degree by assigning a color to a contractor’s historical quality and delivery 
performance in individual Eederal Stock Classifications (ESCs). Red is high risk, yellow 
is moderate risk, and green is low risk. A neutral label is applied to either the 
quality/delivery classification under the following conditions: 1.) if the contractor is a 
first- time offeror for the ESC, 2.) if there is no quality /delivery history available for the 
ESC, or 3.) if available quality /delivery history for the ESC is outside the RYG evaluation 
time frame. The Eogistics Center acts as a clearinghouse with little or no input into the 
supporting data. Each Vendor is notified of its commodity grade and allowed to 
challenge or verify within 30 days. Vendors get detailed information down to line item 
number. Thus far, the Eogistics Center has been challenged three times in court and has 
won. The process goes after value, not low bidders. 

Past Performance Information Retrieval System (PPIRS) 


NAVSEAEOGCEN also administers a web- enabled application, PPIRS, which allows 
the rapid retrieval of contractor past performance information. The data are available to 
all source selection officials across the entire Eederal government. The system is also a 
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central warehouse used to retrieve performance assessment reports received from the four 
recognized Federal report card collection systems, which are: 

- National Aeronautics and Space Administration (NASA) Past Performance Data 
Base (PPDB), 

- National Institutes of Health (NIH) Contractor Performance System (CPS), 

- Army's Past Performance Information Management System (PPIMS), and 

- Contractor Performance Assessment Reporting System (CPARS) used by the 
Navy, USMC, Air Force, DLA and other defense agencies. 

Contract Performance Assessment Reporting System (CPARS) 

The CPARS is a Web-enabled application that collects and manages a library of 
automated contractor performance records. CPARS assesses a contractor's performance 
and provides a record, both positive and negative, on a given contract during a specific 
period of time. Each assessment is based on objective facts and is supported by program 
and contract management data such as: cost performance reports, customer comments, 
quality reviews, technical interchange meetings, financial solvency assessments, 
construction/production management reviews, contractor operations reviews, functional 
performance evaluations, and earned contract incentives. 

Key Observations: NAVSEALOGCEN 

- DoD has a single, consistent logistics program for purchasing hardware and 
materials. 

- The Navy LOGCEN has standardized supply chain evaluation (contractor 
performance) monitoring across the command. 

- The Navy and its prime contractors use receiving inspection information to grade 
sub-tier contractors and suppliers. 

- CPARS provides source evaluation boards access to past performance for large 
contracts. 

- To date, the RYG Program has withstood the scrutiny of courts during three legal 
challenges. 

3.6 Certification Processes 

The submarine Navy definition of “certification” or the “certification process” can be 
summarized as follows: 

- A written statement attesting that the ship or system has met all requirements, 

- Based upon objective quality evidence that documents that deliberate steps were 
taken to comply with requirements, 

- Performed initially upon delivery to the Navy and then sustained throughout the 

9 1 

life of the ship or system. 


NAVSEA 0924-062-0010, "Submarine Safety (SUBSAFE) Requirements Manual. 
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It should be noted that the eertifieation proeess not only applies to SUBSAFE 
requirements, but also involves an overall “material eondition readiness” proeess that 
eertifies that systems, in addition to those eovered by SUBSAFE (i.e., DSS-SOC, 
weapons. Naval Reaetors, ete.,), are ready to support safe operations. 

The following seetions deseribe the SUBSAFE eertifieation proeess within the two 
general eategories of this report: 1.) new design/eonstruetion eertifieation, and 2.) 
maintenanee/modernization or in-serviee eertifieation. 

3.6.1 New Design/Construction SUBSAFE Certification 

New design/eonstruetion eertifieation is primarily eentered on assuring that the 
submarine design, materials, fabrieation, and testing meet all SUBSAFE requirements 
prior to the submarine's delivery to the Type Commander (TYCOM) for Fleet operations. 

The new design/eonstruetion submarine eertifieation proeess was examined for the 
private shipyard, i.e., the Eleetrie Boat ease. Prineipal government partieipants inelude 
SUPSHIP, NAVSEA 92Q SUBSAFE organization, NAVSEA 08 Naval Reaetors 
Program, the submarine program manager, and the ship’s foree (erew). 

The ultimate eertifieation responsibility is vested with the Program Exeeutive Offieer for 
submarines (PEO SUB). PED SUB makes the deeision in eonsultation with the program 
manager and other NAVSEA organizations (05, 92Q, 92T and 08). After new 
design/eonstruetion eertifieation for unrestrieted operations, the submarine is turned over 
to TYCOM who is responsible for sustaining eertifieation throughout its operating life. 

The eertifieation proeess is based on three fundamental elements: 1.) Confidenee that the 
ship designer’s and the shipbuilder’s polieies, proeedures, and praetiees eonform to 
SUBSAFE requirements, 2.) Objeetive quality evidenee that demonstrates eomplianee 
with SUB SAFE requirements during the design and eonstruetion of the submarine, and 
3.) Configuration management during eonstruetion through the re-entry eontrol proeess. 

Confidenee in the polieies, proeedures, and praetiees of the ship designer and shipbuilder 
is aehieved through eonstant surveillanee and audits by the Supervisor of Shipbuilding, 
review of teehnieal produets by the Program Manager and NAVSEA 05, and through 
periodie funetional audits by NAVSEA 92Q. The purpose of funetional audits is to 
eonfirm that an aetivity’s polieies, proeedures, and praetiees eomply with SUBSAFE 
Program requirements, and are “healthy” and eapable of produeing eertifiable design 
produets and eertifiable hardware. 

For eaeh new eonstruetion submarine, eertifieation audits are eondueted. Using a 
struetured audit plan, audit personnel examine the objeetive quality evidenee produeed 
during eonstruetion, ineluding detailed vertieal audits of seleeted eritieal submarine 
eomponents, to eonfirm that SUBSAFE requirements have been met for that partieular 
submarine. The eertifieation audit proeess begins with SUPSHIP who eonduets "modular 
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certification activity" as distinct modules (sections of the submarine) are completed. 
Following the build-up and qualification of modules, a Phase I NAVSEA 92Q SUBSAFE 
certification audit may take place (typically oriented to pressure hull integrity, valves, and 
piping). The final, or Phase II NAVSEA 92Q SUBSAFE certification audit precedes 
Fast Cruise typically by four to six weeks. SUPS HIP internal certification audits are 
conducted prior to both the Phase I and Phase II NAVSEA SUBSAFE certification 
audits. As these audits are conducted, audit cards (non-conformances) are generated by 
both SUPSHIP and NAVSEA 92Q. Certain SUBSAFE audit cards must be adjudicated 
to the satisfaction of NAVSEA prior to the start of Fast Cruise. 

The re-entry control process is invoked with completion of the first pretest inspection on 
a given component or subsystem. From that point forward the hardware is under formal 
re-entry control (REC) configuration management. This configuration management 
continues throughout the remainder of the construction process. Fast Cruise, Sea Trials, 
and ultimately throughout the operational life of the ship. 

Fast Cruise provides an opportunity for the ship’s crew to take sole command of the ship. 
Remaining dockside, but with no outside contact, the crew performs all of its functions 
with respect to operating the vessel. Within a week after completion of Fast Cruise, the 
submarine goes to sea to perform Alpha Sea Trials. Sea trials are designed to 
demonstrate the submarine’s capabilities in small, incremental steps throughout the 
design-operating envelope. The Alpha Sea Trials are followed closely by Bravo Sea 
Trials, then a return to a dry dock facility for necessary work. Following the dry dock 
period, the submarine undergoes Charlie Sea Trials and ultimately is delivered to the 
TYCOMs (COMSUBFANT or COMSUBPAC). The period of time from start of Fast 
Cruise to delivery is typically three months. 

Key Observations: New Design/Construction SUBSAFE Certification 

- The SUBSAFE certification process involves a focused, independent audit 
organization (NAVSEA 92Q) to verify process capability and discipline. This same 
organization conducts detailed vertical audits of selected critical submarine 
components during certification audits. 

- Multiple NAVSEA certification processes operate in parallel to ensure safety and 
operational readiness. 

- REC is invoked early in the certification process. 


22 


Fast Cruise - an integrated operational simulation conducted by the crew alongside the pier. 
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3.6.2 Maintenance/Modernization SUBSAFE Certification 


The SUBSAFE certification audit process applies to all maintenance/overhaul/ 
modernization work performed within the SUBSAFE boundary, from the simplest repair 
on an individual component (e.g., a Technical Availability (TAV)) to a significant 
maintenance/overhaul/modernization availability accomplished in a Naval Shipyard such 
as: 


- A Depot Modernization Period (DMP) 

Typically requires 10 to 12 months to conduct. 

- An Engineered Refueling Overhaul (ERO) 

Typically requires 20 months to complete. 

- A Selected Restricted Availability (SRA)^^ 

Typically takes two months to complete 

An availability greater than six months in duration is typically termed a “Major Depot 
Availability” and requires a NAVSEA 92Q SUBSAFE certification audit. For these 
types of availabilities, NAVSEA certifies to the Type Commander that the work 
performed by the NAVSEA managed field activity (e.g., a Naval Shipyard) meets all 
SUBSAFE requirements. Upon confirmation by the Type Commander that SUBSAFE 
certification requirements have been sustained for those parts of the submarine not 
worked by the NAVSEA managed field activity, the Type Commander reinstates the 
SUBSAFE certification of the submarine. 

An availability of less than six months in duration (an SRA is a typical example) is 
termed a “Minor Depot Availability” and does not require a NAVSEA 92Q SUBSAFE 
certification audit. These types of availabilities do, however, require the quality 
assurance organization within the NAVSEA field activity accomplishing the work to 
perform an audit of the work. The Type Commander also audits the work. Upon 
completion of work, the NAVSEA managed field activity certifies to the Type 
Commander that all of its work meets SUB SAFE requirements. When NAVSEA 92Q 
functionally audits the field activity, a check on the efficacy and health of this process is 
performed. 

The maintenance/modernization SUBSAFE certification process was examined for the 
Portsmouth Naval Shipyard (PNS) case. Principal government participants include the 
PNS SUBSAFE Director and the PNS Quality, Engineering, and Operations 
organizations. Other key players include NAVSEA 92Q, the NAVSEA Shipyard 
Representative’s Office (NSRO), and the Naval Reactors Representative’s Office 
(NRRO). 

The PNS SUBSAFE certification process involves multiple tracks of external assurance 
and audit activity, as well as several internal PNS surveillance and assurance processes. 


It should be noted that some SRAs are completed in private shipyards. 
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PNS uses an internal risk management process called the PreRequisite List (PRL) 
process. This process is a PNS-owned internal risk management process keyed to critical 
milestones ("evolutions") in the maintenance cycle (e.g., dry docking, undocking [or 
flooding the dry dock], Integrated Propulsion Plant testing. Fast Cruise, and Sea Trials). 
Prior to each critical milestone (key event), all shops and codes must certify that work 
and testing needed to support that key event are complete, or have been waived by the 
appropriate technical codes with the approval of shipyard management. 

The shipyard employs a phased certification approach beginning with an Undock Audit 
(focus on hull structure and hull & backup valves), followed by an Integrated Propulsion 
Plant Test Program (IPPTP) Audit (which consists of over 100 Task Group Instructions 
and a selected vertical audit), and finally, a Shipyard Certification Audit (involving 
vertical audits of selected hull cuts, air valves, and sea valves). These audits are 
conducted in advance of the 92Q SUBSAFE audit. At the time of the 92Q audit, PNS 
internal audit cards may be in- work, but 95% of all re-entry control packages must be 
complete prior to requesting the audit be performed. 

The NAVSEA 92Q SUBSAEE audit includes a broad sampling audit to spot-check 
compliance with requirements as well as an in-depth, vertical audit of at least three 
critical components. Examples include pressure hull cuts, emergency main ballast tank 
blow system valves/piping, SUBSAEE seawater system valves/piping, ship control 
system, and complex components such as the 3- inch Eauncher and the Trash Disposal 
Unit. In parallel with SUBSAFE certification activity, the ship’s crew must successfully 
pass a series of Ship Eorce Tests, examining and verifying crew operational capability. 

Prior to NAVSEA concurrence for proceeding to East Cruise, the Submarine Availability 
Completion Manual Appendix A must be submitted to the NAVSEA Program Manager. 
As noted earlier, certain shipyard and/or NAVSEA 92Q audit cards must be closed out 
prior to East Cruise. The squadron, the ship, the shipyard, the program manager, 
NAVSEA 92T, NAVSEA 92Q, and NAVSEA 92 must all concur prior to East Cruise. 

During East Cruise problems may be identified and must be resolved before final 
certification. Carefully scripted Sea Trials are designed to exercise the performance 
envelope ("angles and dangles") where other problems may be identified that also must 
be corrected prior to certification for unrestricted operations. Upon completion and close 
out of all work, testing, and mandatory SUBSAEE audit cards, the final message is sent 
recommending unrestricted operations. 

The in-service certification process not only certifies that SUBSAEE requirements have 
been met, but also includes a “material condition readiness” process that verifies that 
systems in addition to those covered by the SUBSAEE Program are ready to support safe 
operations. 
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Key Observations: Maintenance/Modernization SUBSAFE 
Certification 

- Each individual project superintendent (dedicated 100% to work for a single 
availability) conducts advance planning for compliance with SUBSAFE 
requirements. This approach provides for focus, continuity, and clear accountability. 

- The SUBSAFE certification process involves a focused, independent compliance 
verification organization (NAVSEA 92Q) to verify process capability and discipline. 
This same organization conducts vertical audits of selected critical submarine 
components during certification. 

- Multiple NAVSEA certification processes operate in parallel to ensure safety and 
operational readiness. 

- RFC (re-entry control) is invoked as a vehicle to ensure that any work accomplished 
within the SUBSAFE boundary is planned, executed, and tested in a controlled and 
deliberate manner. 

3.7 Naval Reactors 

NAVSEA 08 Naval Reactors (NR) is the Navy code responsible for all naval ship 
reactors, their prototype and Moored Training Ship plants, and their associated 
radioactive materials. NAVSEA 08 establishes requirements and verifies implementation 
of requirements for reactor design, construction, testing, installation, training, operation, 
maintenance, and disposal. NAVSEA 08 also directs two Department of Energy 
laboratories: Bettis (near Pittsburgh, PA) and Knolls (near Schenectady, NY). 

The success of NAVSEA 08 relies on the selection of the most highly qualified people 
and the assignment and assumption of full responsibility by all members. Training is 
given great emphasis. NAVSEA 08 uses the Challenger accident as a part of its training 
program along with Diane Vaughn's book, “A Challenger Eaunch Decision: Risky 
Technology, Culture, and Deviance at NASA.” NAVSEA 08 points to its safety record 
relative to that of the civilian world’s nuclear activities. The Navy has logged 
126,000,000 miles of safe travel using nuclear propulsion - nearly the distance from the 
sun to Mars. 

A basic tenet of the NAVSEA 08 culture is to make every person acutely aware of the 
consequences of substandard quality and unsafe conditions. Each person is assigned 
responsibility for assuring the highest levels of safety and quality. NAVSEA 08 places 
strong emphasis on mainstreaming safety and quality assurance into its culture rather than 
segregating it into separate groups. The discipline of adhering to written procedures and 
requirements is enforced, with any deviations receiving careful, thorough, formal, and 
documented consideration. 

NAVSEA 08 field offices at shipyards, laboratories, and other major program facilities 
are considered a part of the headquarters organization, and they provide the means for 
facilitating headquarters oversight of facility operations. These field offices are headed 
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by senior personnel from the Washington Navy Yard headquarters who often return there 
after their field duty to fill the highest management and technical positions. 

NAVSEA 08 has integrated the safety process throughout its organization. The 
commanding officers of nuclear powered ships, nuclear support and maintenance 
activities, and NAVSEA 08 field offices communicate directly with the Director on 
issues they are investigating. Issues are confronted while they are small to prevent larger 
problems from occurring. The communication chain is direct and enforced. Corrective 
actions follow a closed loop corrective action process that addresses the root cause of an 
issue, assigns a corrective action, tracks application of the corrective action, and 
subsequently evaluates the effectiveness of that action. Issues are tracked back to 
requirements and training for improvements. NAVSEA 08 tracks and trends concerns 
and initiatives, and it evaluates the practices and processes that outside nuclear 
organizations employ. 

Each organization within NAVSEA 08 is “safety-directed.” There is an escalation 
process that encourages (actually demands) raising issues and minority opinions to the 
“front office” without retribution. 

Key Observations: NAVSEA 08 Naval Reactors 

NAVSEA 08 relies on highly qualified, highly trained people who are held personally 
accountable for safety. 

The NAVSEA 08 management hierarchy (including support management, e.g.. Public 
Communications Director) is technically trained and qualified in nuclear engineering 
and experienced in nuclear reactor operating principles, requirements, and design. 
Problem awareness is essential to high degrees of safety and reliability. NAVSEA 08 
demands that problems be raised to the proper levels when they happen and promotes 
the airing of minority opinions. 

Note: Eurther review of NAVSEA 08 is currently scheduled to take place during phase- 
2 of the NNBE effort. Additional details and key observations regarding 
NAVSEA 08 will be included in the final report. 
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4.0 Context and Opportunities 


4.1 Comparative Context - Differences 

As noted in the introductory section of this report, the submarine Navy and NASA’s 
human space flight programs have a number of factors in common, the most important of 
which is a dedication and commitment to safety while conducting missions of national 
importance in very hostile and hazardous environments. 

However, a number of significant differences (e.g., managerial, organizational, and 
cultural) exist between NASA and the submarine Navy. It is necessary to further examine 
and understand these key differences in detail in order to provide the proper contextual 
background against which the key observations and opportunities developed from this 
benchmarking exchange can be appropriately evaluated. 

Management Approach 

In the SUBSAFE and Nuclear programs, the Navy generally defines both the safety 
requirements “what” and the “how" for critical safety items where the Navy has 
knowledge of what is required to deliver a safe system. Where new technology is being 
developed and the “how” has not been established, the Navy reviews and approves the 
basic approach that is recommended by its contractor(s). Particular emphasis is placed on 
strict compliance with mandatory requirements. 

The flexibility of "how” designers, constructors, and maintainers conduct work only 
exists in systems outside the SUBSAFE and nuclear boundaries. When shown necessary 
through experience or analysis, the "how" can be changed through a formal, well-defined 
process. This applies equally to both the new design/construction and 
maintenance/modemization activities. NASA, on the other hand, has in recent years 
moved toward a number of new and innovative contracting approaches that specify the 
safety requirements “what,” but provide the contractor/subcontractor teams with a level 
of flexibility in determining the “how” with regard to critical and non-critical systems. 

Technical Requirements Authority 

The Navy maintains a central technical requirements authority (including SUBSAFE 
requirements) via NAVSEA 05 separate from individual program and project managers. 
This results in a single set of technical SUBSAFE requirements that are applicable to 
both new construction and maintenance availabilities. Conversely, NASA has a 
distributed network of policy and technical standards documents that are the 
responsibility of various headquarters functional management organizations and center- 
based engineering organizations. Individual NASA program or project managers have 
full engineering and technical requirements authority and establish their own unique 
technical requirements based on higher-level policy guidance. While NAVSEA 08 
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organizationally has project officers (or program managers in NASA terminology) and 
technical directors that report directly to the Director, they both hold responsibility for 
safety. 

Safety Roles 


NAVSEA 92Q has important responsibilities concerning requirements definition, 
conducting analysis, providing direction to field activities, etc. NAVSEA 92Q flows 
down NAVSEA 05 technical requirements. NAVSEA 92Q is responsible for the 
issuance and revision, as necessary, of the SUBSAEE and DSS-SOC manuals, which 
contain the policies, procedures, practices and technical requirements for the respective 
systems. 

NAVSEA 92Q is also the Navy’s primary compliance organization for SUBSAEE and 
deep submergence safety, that is, its function is to assure that policies, procedures, 
practices, and technical requirements are implemented in accordance with SUBSAEE and 
Deep Submergence Systems (DSS) requirements. Compliance assurance is achieved 
through the implementation of a rigorous, no-nonsense functional and certification audit 
process. 

While NAVSEA 92Q has a strong identity as an independent compliance verification 
organization, the NASA Office of Safety and Mission Assurance (OSMA), and its center- 
based counterparts have multiple (and some times conflicting) roles and responsibilities 
that include: 1.) in-line consultant, 2.) in-line direct technical support, and 3.) 
independent compliance verification. 

NAVSEA 08 takes a different approach from either NASA or NAVSEA 92Q. Reactor 
safety is the primary responsibility of all personnel at NAVSEA 08. While NAVSEA 08 
performs functions that are not traditionally associated with reactor safety, all of these 
functions ultimately support reactor safety. Eor example, by providing program 
management for reactor plant work, NAVSEA 08 ensures that pressures from budgets 
and schedules do not impact reactor safety. Eikewise, although there is a separate safety 
organization within NAVSEA 08, it is an independent and equal voice in design and 
operation decisions, and it does not impose after-the-fact safety requirements or 
interpretations. Additionally, it serves as a coordinator, interpreter, corporate memory, 
and occasionally, an advocate for specific capabilities in a system of interlocking 
responsibility in which everyone from the director to the most junior operator is 
accountable for reactor safety. 

NAVSEA 08 has the Naval Reactors Representative's Office (NRRO) perform 
independent compliance verification for various aspects of its responsibilities. In 
addition, headquarters conducts periodic and event-driven inspections and audits of fleet, 
vendor, and other activities, including the effectiveness of lower-tier auditing functions. 
The Nuclear Propulsion Examining Board in each of the Atlantic and Pacific Elects 
performs fleet and facility assessment in close coordination with, and in accordance with 
the requirements and specifications of, NAVSEA 08. 
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Tailoring 


For all submarine programs, the Navy establishes a single set of SUBSAFE technical and 
procedural requirements that are not subject to tailoring by program managers. NASA 
policies allow tailoring and negotiating requirements for individual programs. The 
NASA program manager has the latitude to tailor and negotiate requirements during the 
program formulation phase, as codified in NPG 7120.5B. 

Waivers 


For new construction submarines, NAVSEA does not normally accept waivers to the top- 
level SUBSAEE requirements. In rare cases a waiver to a SUBSAEE technical 
requirement is accepted, and then only when based on sound technical rationale and with 
the concurrence of the technical authority (NAVSEA 05) and the SUBSAEE Office 
(NAVSEA 92Q). Eor in-service submarines, NAVSEA accepts departures from 
specifications on SUBSAEE requirements after careful technical analysis and approval of 
the departure. The departure from specification may or may not have limitations imposed 
upon it (e.g., fix upon return to port, ship limited in depth until fixed, ship’s speed limited 
until fixed, etc.). The Space Shuttle Program does accept waivers to its program 
requirement baseline, however, it employs similar management controls that include 
development of technical rationale, presentation to appropriate review boards, and formal 
management approval. 

Awareness of Risk and Consequences of Eailure 

NASA has relatively few people who have had the opportunity to fly in space, while the 
Navy has had thousands individuals who have been to sea in a nuclear powered 
submarine. There are a significant number of people (both military and civilian) in the 
Navy whose personal safety has been dependent on the safe operation of a submarine. In 
addition, many of the submarine senior managers, designers, and shipyard workers, both 
contractors and NAVSEA personnel, routinely ride onboard submarines during Sea Trials 
following construction and maintenance availabilities. 

Certification Processes 


Before each submarine completes a maintenance availability, the Navy conducts 
horizontal and vertical audits in the SUBSAEE boundaries to verify compliance with 
safety requirements. These independent compliance verification audits impact the 
workforce by increasing emphasis on the quality and fidelity of work execution. This 
scrutiny serves to elevate individual motivation and attention to details. 
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A vertical audit, as accomplished by NAVSEA 92Q, is a "core drill" investigation of 
selected critical components in which compliance with requirements is tracked to the 
lowest level, such as described in the following notional example: 

Who installed the component? 

Who inspected the work? 

Where are the individuals' training and certification records? 

Where is the test data for the component? 

Where are the certification test records (objective quality evidence)? 

Who performed the tests, and what were the individuals' qualifications? 

Who witnessed the test? 

What is the material heritage? 

Where are the lot acceptance fracture toughness test data? etc. 

NAVSEA 08 conducts similar audits before key events such as initial reactor criticality. 

In addition, in both the SUBSAEE and NAVSEA 08 approaches, background processes 
(e.g., audits and on-scene observations) continuously evaluate critical processes and 
correct any problems with work accomplishment or documentation required to validate 
the acceptability of prior work. 

NASA does not routinely conduct vertical audits of selected critical components as a pre- 
launch milestone in the Space Shuttle Program, nor does NASA routinely conduct 
vertical audits for other space vehicles. However, NASA does conduct detailed 
investigations similar to vertical audits in the case of specific anomalies that may be 
identified in pre- launch processing or as a result of data from previous flights. 

Safety Awareness Training 

As discussed in Section 3, NAVSEA uses the USS THRESHER loss as a powerful 
training tool, emotional lesson, and organizational motivator - all with the purpose of 
helping the Navy to ground and sustain its safety culture. As a part of this training, 
personnel are required to watch a video, listen to the audio of the collapse of a 
submarine’s hull and compartments, and are reminded that people were dying while this 
occurred. The NAVSEA 08 Naval Reactors Program has never experienced a reactor 
accident, but nevertheless includes training based on lessons learned from program 
experiences. NAVSEA 08 also looks outside its program for lessons learned from events 
such as Three Mile Island, Chernobyl, and the Army SE- 1 reactor. NASA has a similar 
painful memory with the Challenger loss, but NASA has not yet incorporated it as part of 
a formal, institutionalized training tool for emphasizing safety and motivating personnel. 
The Navy, however, does incorporate the Challenger loss into its safety awareness 
training. 
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Levels of Contractor Insight 


As demonstrated by the Navy’s comprehensive approach to SUBSAFE, weapons 
systems, nuclear reactor systems, and its certification process, the Navy imposes a high 
level of "insight" into its contractor activities to assure safety and program success. 
When compared to NASA’s insight continuum^"^, the Navy implements contractor 
“insight” to a penetration level of 3 or 4. The NASA Space Shuttle and International 
Space Station Programs implement comparable penetration levels; however, preliminary 
NASA planning discussions for the Orbital Space Plane (OSP) have involved 
consideration of lower levels of insight. 


In the language of the NASA Insight Continuum - see below: 

MSFC HB 3173, page 57: Definitions of "Insight Level” 

Technical Penetration Level 0 - No Penetration 

Accept performing organization’s tasks at face value (based on assessment that no penetration is 
required) 

Contractor develops and implements verification plan 

Technical Penetration Level 1 - Low Penetration 

Participate in reviews and technical interchange meetings and assess only the data presented 
Perform periodic audits on predefined processes 

Chair board or serve as board member, or RID writer, at a formal review 
Participate in resolution and closure of issues 
Review verification plan and its implementation 

Technical Penetration Level 2 - Intermediate Penetration 

Perform low penetration tasks with addition of daily or weekly involvement to identify and 
resolve issues 

Review verification plan, its implementation, and selected verification closure data 

Technical Penetration Level 3 - In Depth Penetration 

Perform all tasks at the intermediate penetration level 
Perform methodical review of details 

Develop independent models to check and compare vendor data, as required 

Review verification plans and their implementation and concur in all verification closure data 

Technical Penetration Level 4 - Total Penetration 

Perform a complete and independent evaluation of each task 

Perform independent review of all verification documentation (including closure data) and witness 
verification testing 
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4.2 Potential Opportunities 


The following discussion offers potential improvement and enhancement opportunities 
for NASA to consider as its human space flight programs continue to evolve and 
advance. These opportunities are not meant to imply inadequacy of current NASA safety 
processes, but rather to support areas where the submarine Navy's experience could 
benefit NASA human spaceflight programs. 

Opportunities have been organized into three topical groups: 

Group I: Requirements and Compliance 

Group II: Lessons Learned & Knowledge Retention 

Group III: Process Improvement 

Group I; Requirements and Compliance 

The Navy manages portions of programs that affect safe operations using a simple, two- 
part management philosophy based on 1.) clear and realistic requirements definition and 
2.) independent verification of compliance. Individual NASA programs create 
requirements baselines, but have historically found it necessary to waive many program 
requirements that cannot be met, while mitigating the risks associated with those waivers 
through other means. 

Opportunity #1.1: Functional Safety Requirements for Future Human Rated Space 
Systems 

The NAVSEA ship specification (including the SUBSAFE and DSS-SOC manuals) is the 
de facto submarine human rating design requirements document. This “ship spec” 
reflects best practice that has evolved by incorporating known and postulated risks, 
knowledge of current technology and design practices, results of tests and actual 
operations, and lessons learned. 

NASA may wish to explore a similar approach in developing a corporate- level human 
rating safety guidance document for future NASA human space flight programs. This 
safety requirements document would be based on the functional design attributes of 
current and past NASA human- rated space flight systems. The document would define 
specific functional safety requirements (e.g., functional redundancy, intervention 
capability, abort/escape capability), and it would require formal and rigorous audits and 
assessments to verify implementation of and compliance with those requirements. 

Under this approach NASA program managers would not be allowed to unilaterally 
waive these requirements. Disposition of waiver requests pertaining to specific, critical 


The team notes that a draft policy guidance document is currently in-work, entitled "Human Rating 
Requirements for Space Flight Systems" . 
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safety functional design requirements might be the responsibility of a (new) corporate- 
level, NASA HQ Human Space Flight Safety Review Board or equivalent forum. 


References: Key Observation Sections 3.3.1 and 3.3.2 

Opportunity #1.2: NAVSEA Model for Compliance Verification Organization 

The NAVSEA model involves a separation of program authority, technical authority (i.e. 
binding technical policy requirements), and independent compliance verification for the 
elements essential to safe submarine operations. NASA may wish to explore the 
potential benefits associated with a variant on this model. While it may not be practical 
for NASA to achieve centralized technical authority as accomplished in the NAVSEA 
model, NASA may wish to adopt the concept of a centrally controlled, separately funded, 
independent safety compliance organization (independent of program management 
authority) to verify the implementing organization’s functional capabilities, and to verify 
compliance with the program/project baseline safety and mission assurance requirements. 

The NAVSEA 92Q analog (organization, staffing, functions, processes, etc.) may be 
useful for NASA to consider in moving toward an independent safety requirements 
compliance verification organization for all programs involving human space flight 
operations, including future NASA human rated space systems (e.g.. Orbital Space Plane) 
currently under development where innovative contracting approaches may be employed. 


References: Key Observation Section 3.2.2 


Opportunity #1.3: Compliance Verification for New NASA Human Rated 
Programs 

A central theme in the NAVSEA model for success revolves around the application of 
high levels of government oversight (compliance verification and approval authority) into 
contractor activity for all critical submarine processes. Nowhere is this approach more 
pronounced, and the evidence of success more apparent, than in the nuclear reactor 
program area. The Navy’s insistence on full government insight and individual 
ownership, accountability, and attention to detail (started under Admiral Rickover) has 
ensured unparalleled success in nuclear reactor safety. 

As NASA considers the levels of oversight that it will impose on its new human rated 
space flight programs and potential future nuclear propulsion programs, a further 
understanding of the Navy’s approach may be appropriate. Specifically, the recent 
(preliminary) program management deliberations concerning Orbital Space Plane 
Program "level of insight" (see footnote on p.54) may provide an ideal opportunity for 
further investigation and comparison. 
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Group II: Lessons Learned/Knowledge Retention 


The NAVSEA centralized technical authority approach directly facilitates institutional 
lessons learning. As pointed out in observations in sections 3.2.1, and 3.4.1, NAVSEA 
mitigates risk through continuous learning and improvements to design based on 
extensive operational experience. This engineering/technology knowledge base is a 
central element in the ongoing success of the Navy submarine program. This success is 
based on a stable, central organization to document operational experience (NAVSEA 
05) and establish technical requirements. 

Opportunity #2.1: Lessons Learned Management at NASA^^ 

The recent GAO report "Survey of NASA’s Lessons Learned Process" highlighted 
fundamental weaknesses in the collection and sharing of lessons learned in NASA by 
program and project managers. 

Recognizing current efforts to improve the NASA lessons learned process, NASA may 
wish to move forward with an expanded initiative, under the leadership of the NASA 
Chief Engineer and center-based engineering organizations, to build (future) and 
reconstruct (past) project management, engineering, and technology narrative histories 
for NASA programs and projects. The development of a viable NASA lessons learned 
knowledge-base (not database) would become the basis for requirements development for 
future NASA space and aeronautical systems. 

Such an effort would begin with development of a consistent taxonomy or framework for 
information acquisition and analysis. Using the framework, the effort would begin 
developing histories for recent NASA programs and projects, eventually reaching back to 
Apollo, Gemini, and Mercury. 

It would also be necessary to strengthen policies related to the use and implementation of 
lessons learned. A top-level policy document could be complemented by an 
implementation level policy guidance document providing practical guidance on how to 
approach the subject of lessons learned within the program/project environment. 


References.'Key Observation Section 3.2.1, 3.2.4, 3.4.1 


Opportunity #2.2: Lessons Learned Training 

NASA may wish to explore the possibility of developing safety/mission success lessons 
learned training courses for small groups of trainees based on noted NASA failures 
including the Challenger (STS 51L) loss, the 1967 Apollo 1 (Apollo 204) fire, 1970 
Apollo 13 mishap, the 1986 (manufactured) Hubble Space Telescope mirror incident, and 


The opportunities identified in this section seek the same outcomes embodied in the NASA "Strategic 
Human Capital Plan" pillars and goals (#6 and #7). 

“NASA Lessons Learned Survey Results,’’ GAO-01-1015R Survey of NASA’s Lessons Learned 
Process, September 5, 2001. 
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the more recent Lewis Spacecraft, Mars Climate Orbiter, and Mars Polar Lander mishaps. 
Consider required annual participation by selected NASA employees and 
program/project management teams (including contractors) in one or more training 
sessions. Also consider offering training sessions based on the NAVSEA 08 "Challenger 
Launch Decision" (Diane Vaughn) training seminar and the USS BONEFISH Fire 
accident investigation videotape. Space systems, submarines, and other transportation 
modalities (airplanes, ships, trains) can be addressed in an integrated curriculum that 
includes the study of multi-modal failures in complex, tightly coupled systems. 


References.'Key Observation Section 3.2.4 


Opportunity # 2.3: Knowledge Retention (FTE Ceiling Authorization for Hiring)^^ 

During the late 1980’s and early 1990’s, NAVSEA faced a critical loss of engineering 
and management skills as a result of both imposed downsizing and impending aging 
work force issues. NAVSEA's approach for retaining critical knowledge was to employ a 
mentoring approach. In order for this approach to work, it was necessary for NAVSEA 
to obtain approval to increase its hiring ceiling (not its overall budget) allowing them the 
flexibility to hire young engineers while retaining the experienced engineers necessary to 
mentor and provide essential knowledge transfer. NASA may wish to investigate the 
potential benefits of adopting this approach. 


References: Key Observation Sections 3.2.5 


Group III: Process Improvement 

NASA may wish to consider the following opportunities to implement several broad- 
based process improvements. 

Opportunity #3.1; NAVSEA Logistic Center 

NASA has an opportunity to leverage the extensive vendor quality history database 
available at the NAVSEA Eogistics Center to assist in supplier selection and evaluation. 
The development and application of this database is embodied in the following processes 
and programs: 

Product Data Reporting and Evaluation Program 

Eevel 1/SUBSAFE Verification System (may not be not applicable to NASA) 
Supplier Audit Program 
Red/Yellow/Green Program 

Contract Performance Assessment Reporting System, and 
Past Performance Information Retrieval System (PPIRS). 

NASA is currently participating in the PPIRS. Discussions are underway to determine 
the feasibility of obtaining access for the NASA quality assurance community to these 


Opportunity may complement the July 2002, "NASA Human Capital Legislative Proposal” by providing 
a way to train and mentor new employees enabling retention of corporate knowledge. 
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valuable resources. This opportunity will provide assists NASA managers in selecting 
qualified, proven suppliers and vendors. 


References: Key Observation Sections 3.5.3 


Opportunity # 3.2: Software Contracting Approach 

Based on the ship specifications passed down from the Navy, the prime contractor creates 
detailed software specifications. The prime contractor then passes down very detailed 
procurement specifications to its subcontractors. To further assure that the work is being 
completed correctly, each organization has a group located at the next lower level to 
oversee and assure the quality of the work in progress. NASA should consider reviewing 
future software contract language and structure to insure an appropriate level of 
specification detail and implementation assurance consistent with the NAVSEA 
approach. 

NASA also may wish to consider the NAVSEA approach of requiring standardized 
software development and assurance clauses in REPs. In addition, requiring the use of a 
standardized template for software development plans provides an important extra step in 
ensuring software safety. 


References: Key Observation Sections 3.4.2 


Opportunity # 3.3: Human Factors 

NAVSEA has recently established a human factors organization (NAVSEA 03) to 
emphasize the life-cycle human/system interface. NASA may wish to consider 
collaboration with NAVSEA 03 to develop possible human/system interface technical 
standards, policies, and processes for NASA. 

Evaluate how mission goals, function analysis, task analysis, and maintenance/operation 
tasks were developed and modeled in the VIRGINIA Class. Apply lessons learned from 
the VIRGINIA Class and NAVSEA 03 studies to the NASA Orbital Space Plane (OSP) 
program. Include human/system interface as an integral part of total OSP vehicle design 
to optimize system performance, manning, and life cycle cost. 


References: Key Observation Sections 3.4.2 


Opportunity # 3.4: Quantitative Methods 

NASA can improve its use of historic reliability and operational performance data, and 
the use of lessons learned from accident and mishaps, by centralizing this information 
and incorporating it into an Agency risk and reliability database to support design and 
risk assessment teams. 


References: Key Observation Sections 3.4.2 
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4.3 RationaleA^alue of Potential Opportunities 


Table 4. 1 provides a summary of the opportunities identified in this interim report and a 
brief description of the rationale and/or potential value. It is anticipated that other 
opportunities will emerge as the NNBE enters its second phase. 


Table 4.1 RationaleA^alue of Potential Opportunities 
NASA/Navy Benchmarking Exchange Interim Report 

Opportunity Description 

Rationale/Value 

Requirements & Compliance 

1.1 Functional 
Safety 

Requirements for 
Future Human 
Rated Space 
Systems 

Enhance current NASA efforts to 
develop corporate- level human rating 
functional design requirements by 
adding the NAVSEA evolutionary 
(successive ship class specification) 
dimension. 

Establishes a clear, 
executable requirements 
baseline for future 
human rated space 
systems 

1.2 NAVSEA 
Model for 
Compliance 
Verification 
Organization 

Establish, for human rated space 
systems, a centrally controlled and 
separately funded independent safety 
compliance organization to verify the 
implementing organization’s 
functional capabilities, and to verify 
compliance with the program /project 
baseline safety and mission assurance 
requirements. 

Approach creates a 
clearly independent, 
unambiguous 
compliance assurance 
organization. 

1.3 Compliance 
Verification for 
new NASA 
Human Rated 
Programs 

Review current NASA planning 
versus NAVSEA approaches for 
compliance verification to be 
employed for the Orbital Space Plane 
program and potential future nuclear 
propulsion programs. 

Assures that contract 
surveillance and 
compliance verification 
methods have a scope 
and rigor consistent with 
the goal of mitigating 
manageable risks in 
human space flight 
systems. 
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Table 4.1 RationaleA^alue of Potential Opportunities 
NASA/Navy Benchmarking Exchange Interim Report 


Opportunity Description 

RationaleA'alue 

Lessons Learned & Knowledge Retention 

2.1 Lessons 
Learned 
Management at 
NASA 

Move forward with current NASA 
planning to implement an aggressive 
NASA lessons leamed/knowledge 
management initiative, under the 
leadership of the NASA Chief 
Engineer. Using NAVSEA 
“template” combine continuous 
learning with technical authority 
roles. 

Eacilitates requirements 
development for future 
NASA space and 
aeronautical systems. 
Enables programs to 
avoid past mistakes. 

Provides for centralized 
management and 
ownership of lessons 
learned within the 




engineering 

management/systems 

engineering 

organization. 

2.2 Eessons 
Eearned Training 

Explore the possibility of developing 
safety/mission success lessons 
learned training courses for small 
groups of trainees based on noted 
NASA failures including the 
Challenger (STS 5 IE) loss, the 1967 
Apollo 1 (Apollo 204) fire, 1970 
Apollo 13 mishap, the 1986 
(manufactured) Hubble Space 
Telescope mirror incident, and the 
more recent Eewis Spacecraft, Mars 
Climate Orbiter, and Mars Polar 
Eander mishaps. 

Provides motivation and 
reinforces safety culture 
and discipline within 
NASA workforce. 

Provides greater 
understanding of 
failures in tightly 
coupled, complex 
system. 

Reinforces the need to 
address all areas of 
potential critical failure. 

2.3 Knowledge 
Retention (ETE 
Ceiling 

Authorization for 
Hiring) 

Obtain approval to increase its hiring 
ceiling (not its overall budget) 
allowing the flexibility to hire young 
engineers while retaining the 
experienced engineers necessary to 
mentor and provide essential 
knowledge transfer. 

Provides means to retain 
corporate knowledge. 

Enables mentoring of 
new employees. 
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Table 4.1 RationaleA^alue of Potential Opportunities 
NASA/Navy Benchmarking Exchange Interim Report 
Opportunity Description RationaleA'alue 


Process Improvement 

3.1NAVSEA 
Logistic Center 

Leverage the extensive vendor 
quality history database available at 
the NAVSEA Logistics Center to 
assist in supplier selection and 
evaluation. 

Assists in selecting 
qualified, proven 
suppliers and vendors. 

3.2: Software 

Contracting 

Approach 

Review future software contract 
language and structure to insure an 
appropriate level of specification 
detail and implementation assurance 
consistent with the NAVSEA 
approach. 

Ensures that appropriate 
software assurance 
processes are 
implemented throughout 
the software 
development life- cycle. 

3.3: Human 
Factors 

Collaborate with NAVSEA 03 to 
develop possible human/system 
interface technical standards, 
policies, and processes for NASA. 
Evaluate how mission goals, function 
analysis, task analysis, and 
maintenance/operation tasks were 
developed and modeled in the 
VIRGINIA Class. 

Provides the opportunity 
for NASA and Navy to 
exchange lessons 
learned as each 
organization moves to 
provide additional focus 
on human factors in 
system program/project 
life cycle. 

3.4: Tools/ 

Quantitative 

Methods 

Improve NASA databases containing 
historic reliability and operational 
performance data, by centralizing 
information into an Agency risk and 
reliability database. 

Enables use of historic 
reliability and 
operational performance 
data, to support design 
and risk assessment 
teams 
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Additional Reference Material: 


- “Alternate Trajectories: Options for Competitive Sourcing of the Space Shuttle 
Program,” RAND Corporation Executive Summary Report, Nov 6, 2002, 

http://www.rand.org/scitech/stpi/NASA/nasaExecSum final.pdf . 

- “Report by the International Space Station (ISS) Management and Cost Evaluation 
(ICME) Task Eorce to the NASA Advisory Council,” A. Thomas Young (Chairman), 
et ah, November 1, 2001. 

- “International Space Station Independent Implementation Review, Presentation to the 
Code M Program Management Council,” NASA, December 11, 2001. 

- “Enhancing Mission Success - A Eramework for the Euture, A Report by the NASA 
Chief Engineer and the NASA Integrated Action Team,” NASA, December 21, 2000. 
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Appendix A: Initiating Letter and Attachment 


Office of ttie Adrnlnistnitor 



June n. 2U(i: 


The Honnrahlc (lOTiti'n K. I nglaniJ 
Setnelars ol tl:e Na\ _. 

WiishitiKtun. DC 20350-1 iXHi 


lieji Mi, Sci>it3r> 




The Na\y's nuelcai submarine pruurain and S\SAT Itutnan space thghl prognim ha\c 
much ID common Both programs have misstoos that are viully important lu the Nation. 
Both u .%0 htchly complex s;." terns and operate m challengmc pliyxiciil cnvironmeiiM Most 
importantly, both programs arc uitally devoted und committed to safety, 

Tixlir,. Space Shuttle and Inicmatmnal Spa.x Station pnigram mana^eni are 

facing many challenges, including maintaining product quality^ und safety, accomplishing 
rcL|iiircd performance itnd safety upgrades, ard matnlaming a skilled and motivated 
workforce in ihe I'ace of budgci and schedule pressures. Hiese issues are wcii undersiood 
by the Navy‘s nuclear submarine program managers who fn»-cd similar chiillcngcs dnnng 
a downturn in production in the early IWii's 

F believe that .NAS.\ can leant much from the Navy‘s experience To this end, 1 
piopose ilui N.\S \ s Jilicc oi Safety and .Mission .Assurance, in coonlinauon with the 
N'AS.A Aerospace Safety Advisory Panel (ASAP), undertake a benchmarking study of the 
Navy 's nuclear suhirunne programs My gixil r- 1*' develop aset of lesAoas learned llul 
can be applieil to benefit NAS \ \ human .space flight programs 1 have enclosed a draft 
irnplemcniation plan that outlines how we would like to structure tins study. Of course, 
this drali plan is oi>en to turtlicr refinement and negotiations with your staff 

U rcspiectfully, seek your .support <md endorsement of this activity and welcome ;.<iur 
suiitrstiotis lor an app - priatc point of contact and stippon personnel who couiJ assist my 
staff m planmug and conducting this bcndimaffong study 

■_ ordially. 


-ly. 

Enclosure v>T/' 


3ean fVKe-cfe 
Administrator 
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Preliminary Implementation Plan: NASA Benchmarking Review of Nuclear Navy 
Safety and Mission Assurance Processes 


Background 

The Administrator recognized similarities between human spaceflight and nuclear submarine 
programs. Both operate in extreme environments, are composed of many complex subsystems, 
and must maintain the very highest levels of safety and reliability in order to perform their missions. 
While at OSD and serving as SECNAV, Mr. O’Keefe observed the Navy's ability to operate safely 
and effectively in resource constrained and declining production environments. Furthermore, as 
NASA explores the application of nuclear propulsion and power for space exploration, lessons 
learned from the Navy nuclear safety program could be beneficial. Thus, given the current 
management challenges NASA is facing, Mr. O'Keefe recognizes that NASA may benefit from a 
critical examination of the engineering management, safety, and mission assurance, approaches 
employed by the nuclear Navy. 

Objective 

Benchmark Navy nuclear submarine program with focus on safety and mission assurance policies, 
processes, accountability, and control measures that can assist NASA in meeting current human 
space flight challenges. Document and disseminate findings (lessons learned) to enable multi- 
level NASA organizational learning. 

Approach: 

1. Send a letter from the NASA Administrator to the Secretary of the Navy proposing the 
benchmarking review. 

2. In parallel, arrange meeting between Flag/Senior Executive principals in order to scope the 
benchmarking activity and to agree on next steps. Identify prospective individuals (managers, 
process owners, operators) to interview during the assessment. 

3. Conduct a series of technical interchange meetings and on-site visits to gather information and 
obtain insight into the associated processes of interest to NASA. 

4. Complete factual review draft report for Navy review 

5. Deliver final report to the NASA Administrator. 
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Themes 


Overarching Safety and Risk Management Approach 

Understand the nuclear Navy top-down approach for ensuring accountability and control in 
safety critical areas. Understand the flow-down of top level safety philosophy and 
requirements. Understand the overarching risk management posture or logic employed in 
making decisions concerning competing and often conflicting program dimensions of cost 
control, schedule, mission capability, and safety. 

SUBSAFE Program 

Understand the cultural dimensions of the SUBSAFE Program. Acquire a process-level 
understanding of safety, risk management, and mission assurance processes within the 
Navy nuclear submarine SUBSAFE Program. Understand the policies, procedures, 
practices, and processes for 1.) definition of requirements, 2.) verification of ongoing 
implementation, and 3.) certification of operational readiness. Clearly identify 
organizational accountability and responsibilities for SUBSAFE implementation. 

Change Control Processes 

Understand the authority and relationships between various control boards and processes 
responsible for hardware design, software design, manufacturing processes, 
test/certification, and operations. Understand the role of systems engineering authority 
within the Trident program for integrating and managing change. Understand the 
organizational responsibilities and processes employed for configuration management of 
requirements baselines. 

People in the Process 

Capable critical processes require capable people. Explore and understand the policies, 
processes, and controls implemented for assuring the ongoing capability of people in 
critical processes. Topics to include hiring, training, certification, and retention of critical 
skill individuals. Need to understand approaches for ensuring a reliable (human reliability) 
and capable workforce in terms of health, stress, overtime, extended duty, physical, and 
psychological work environment. Need to understand the administrative or management 
approaches implemented to ensure that critical processes maintain necessary staffing 
levels to function in a stable, capable, and controlled manner. 


Scope 

Potential Trident Submarine Program focus - similar in operational age and safety criticality to 
Space Shuttle program. Trident has a similar technology era pedigree, similar need to evolve 
(mission capability and safety upgrades) over time, and similar high reliability requirements. In 
order to facilitate real-time assessment of change control processes it is proposed that the review 
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narrow the focus to the SSGN Conversion Program, which is an ongoing activity to retrofit Trident 
D5 missile tubes to support Tomahawk cruise missile and special operations capabilities. 

Locations/ facilities 

Potential SSGN Conversion Program site visits might include: 1.) Naval Sea Systems Command 
in the Washington Navy Yard, 2.) Kings Bay, GA, and 3.) Electric Boat in Groton, CN. 

Phase of Life-cycle 

- Development (primarily upgrade development) 

- Manufacturing/Manufacturing verification 

- Pre -Operations Integration & Test/Certification 

- Operations/Maintenance 

Security Issues 

An area requiring close coordination. Acquire a dedicated security liaison officer to support team 
activity. What level of clearance is required by the team members? 


Timeline 

Begin as soon as possible (depending on Navy response) and complete final report this summer. 

Review Team Staffing (team composition/lead) 

Proposed NASA/HQ/OSMA Programmatic lA Team plus at least one participant/observer from the 
Aerospace Safety Advisory Panel (ASAP). 
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Appendix B: Summary of Key Observations 



3.1 

SUBSAFE Program 


The SUBSAFE concept addresses specific flooding/recovery hazards and 
places special life-cycle emphasis on controlling those hazards. Other hazard 
control boundaries exist on a submarine as shown in figure 2.2. 

There is clarity, uniformity, and consistency of submarine safety requirements 
and responsibilities. Tailoring by program managers is not permitted without 
NAVSEA 05 and 92Q approval. 

There is a strong, independent “audit to requirements” assurance organization. 
There is a community- wide (contractor and government) understanding of 
SUBSAFE requirements and a commitment to compliance. 

The SUB SAFE Program and implementing organization (NAVSEA 92Q) are 
relatively immune to budget pressures - no certification, no submarine. 

There is a strong, community- wide (contractor and government), continuing 
emphasis on safety through periodic systematic examination of previous 
failures and incidents. 

Annual SUBSAFE training is a requirement for the NAVSEA Headquarters 
submarine community. 

3.2 

Management and Organization 


3.2.1 

Technical Authority 

The NAVSEA organizational structure provides a capable, centralized, 
independent, technical authority that is responsible for developing and 
documenting technical requirements and standards, and providing 
requirements clarity and accountability. 

There exists community- wide acceptance that NAVSEA 05 is the technical 
requirements owner. 

Any delegation of technical authority from NAVSEA 05 to implementing 
organizations is clearly documented. 

The centralized technical authority provides a powerful means to capture, 
document, and use lessons learned to impro’se future ship designs. 

3.2.2 

Independent Compliance Verification Organization 

The NAVSEA management organizational structure provides a highly capable 
and independent safety compliance verification/assessment organization that 
serves as a key managemeit control for SUBSAFE certified activities. 

The compliance verification process is the responsibility of an entity separate 
from program management and the operators (the Fleet Type Commanders) of 
submarines. 

Audit activities draw on expertise distributed across the Navy laboratories, 
shipyards, and other field activities. 
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Summary of Key Observations 


3.2.3 Safety Management Philosophy 

Contractors are not given latitude in meeting SUBSAFE and other critical 
safety system requirements because the leverage gained in the areas of 
technical excellence and risk mitigation achieved through the many 
generations of submarine specifications would be lost with such an approach. 

3.2.4 

Cultural Attributes 

Safety is central to the culture of the entire Navy submarine community, 
including builders, designers, maintainers, and operators. 

- In addition to the loss of the USS THRESHER and the USS BONEFISH fire, 
the Navy has made extensive use of the Challenger mishap in its safety 
training. 

3.2.5 

EB Restructure & Downsizing 

The Navy worked closely with EB to ensure that a logical and structured 
approach was employed to downsize the workforce. There was no "magic 
pill" for what was a painful downsizing process. 

Changes that were made at EB were driven by the need to survive as a 
business unit within the General Dynamics Corporation. Union and corporate 
management worked together to determine the optimum mix of competencies, 
seniority, and management for the restructure and survival of the company. 
Having clearly defined, well- documented technical and procedural 
requirements assisted the Navy and EB during the restructuring and 
downsizing transition. 

3.2.5 

NAVSEA Restructure & Downsizing 

NAVSEA conducted a logical and structured downsizing approach that 
identified core competencies and the engineering workforce requirements 
necessary to assure continued support of critical defense technologies. 
NAVSEA overstaffed its engineering skill sets with recent college graduates 
to allow the skilled work force to train new hires while maintaining core 
competencies. 

3.3 

Safety Requirements 


3.3.1 

Safety Requirements Management 

Critical safety requirements and implementation methods are clearly defined. 
Critical safety requirements are protected, and program managers cannot tailor 
them or trade them against other technical or programmatic variables. 

3.3.2 

Submarine Safety Design Criteria 

There does not exist a single (stand alone) document that proscribes NAVSEA 
human rating design safety criteria or standards. 

The existing operational design attributes, compiled in the most recent ship 
specification, represent the de facto, evolved human rating design standard. 
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Summary of Key Observations 


3.4 


Implementation Processes 


3.4.1 Design Approach 


New ship designs are evolutionary. Lessons are learned and applied from one 
submarine class to the next. 

The VIRGINIA contract partnership agreement (EB and Newport News 
Shipbuilding) enabled a strong degree of collaboration. 

The newest design class, VIRGINIA, employs design for manufacturability 
(six- sigma) concepts. NAVSEA and EB consider the approach, referred to as 
design/build, as critical to achieving affordability. 

Early and continual collaboration between government and contractors 
reduces programmatic risk by emphasizing and strengthening design for 
manufacturability . 


3.4.2 Processes/Tools 


Quantitative Methods 


Traditional safety/hazard assessments are used and rely extensively on the vast 
historic data (design/build/test/maintenance and operational experience) of the 
US Navy Submarine Program. 

Reliability and maintainability assessments are performed according to 
traditional DOD methods and also rely heavily on the vast historic data of the 
program. 

Probabilistic Risk Assessment does not appear to be used at any significant 
level. 

Risk assessment is applied qualitatively/categorically using a DOD-prescribed 
5x5 likelihood versus consequence matrix that is similar to the one used by 
NASA. 


Software Development 


The VIRGINIA Class represents the first NAVSEA application of fully fly- 
by- wire technology for ship control system design; accordingly, its software 
management and control processes are evolving. 

The current approach represents a comprehensive and conservative 
management approach to assuring software capability and fidelity. 

EB implements a very rigorous software contracting process that assures that 
safety requirements are appropriately Bowed-down and met at each level (i.e.. 
Navy to EB to subcontractors). 

EB use of software-based design tools (e.g., CATIA) has helped them 
maintain and communicate its work within a distributed engineering design, 
manufacturing, and maintenance environment. 


Human Factors 


NAVSEA systematically considers life-cycle human/machine interface 
requirements in the design of submarines. 

NAVSEA 03 - “Human Systems Integration in System Design” was created 
to ensure that human/system interface is an essential element in total ship 
systems engineering. 

Human Eactors Engineering requirements, unique to SUBSAEE and deep 
submergence systems, are included in the SUBSAEE and DSS-SOC manuals. 
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Summary of Key Observations 


3.4.3 

Work Planning and Authorization Processes 
(Maintenance/Modernization) 


SUBMEPP 

SUBMEPP advanced planning is a key element in ensuring Submarine Safety. 
This approach provides management, engineers, and technicians ample time to 
evaluate task steps and detect/correct procedural errors, thereby increasing the 
likelihood that activities will be performed safely. This approach also ensures 
that needed parts, tools, and personnel are available when the task will be 
performed. 

SUBMEPP is the “detail” organization (e.g., checking tolerances and factors 
of safety) assuring that the right maintenance work is done when it needs to be 
done and in the right way. 

SUBMEPP advanced planning also provides valuable data that allows the 
Navy to make informed decisions concerning workforce staffing, training, and 
retention. 


SHAPEC 

SHAPEC was created to place the detailed planning for 
maintenance/modemization availabilities for the SSN 688 Class of ship under 
one entity. This process has reduced cost, complexity, and redundancy of 
efforts, and it has made available the best practices, lessons learned, and most 
efficient production processes available to all public shipyards performing 
maintenance activity. 

3.4.4 

NAVSEA Fleet Modernization Program 

NAVSEA manages configuration of modernization efforts via use of a well- 
documented SHIPAET process. 

Priority is placed on safety, reliability, maintainability, habitability, and 
environmental compliance. 

The EMP is oriented toward long-term upgrade planning (5-10 year horizon). 

3.5 

Compliance Verification Processes 


3.5.1 EB Quality 

SUPSHIP serves as a strong, independent, government quality assurance 
oversight organization located at the contractor’s facility. 

NAVSEA 08 represents a second strong, independent, government quality 
assurance oversight organization also located at the contractor's facility. 
Multiple assurance processes (EB, SUPSHIP, DCMA, and NAVSEA) extend 
down into the EB supply chain. 
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Summary of Key Observations 


3.5.2 PNS Quality 

PNS quality processes are integrated into maintenance availability activities in 
order to achieve maximum assurance of compliance. 

Discipline and individual worker integrity are considered critical to the quality 
assurance process. 

Honest mistakes are not punished. Not reporting mistakes can subject a 
worker to discipline because the Navy emphasizes full disclosure in the 
reporting process in order to be able to promptly identify and fix problems. 
Workers are made aware of the consequences for failure to report. 

The Project Management approach provides workers with a sense of 
continuity and pride of ownership for the work to be performed on a given 
submarine. 

The Navy uses a standardized trouble reporting system across all 
shipyard/repair facilities. This standardized process helps facilitate the 
sharing of significant problems with other shipyards. 

3.5.3 

NAVSEALOGCEN 

DoD has a single, consistent logistics program for purchasing hardware and 
materials. 

The Navy LOGCEN has standardized supply chain evaluation (contractor 
performance) monitoring across the command. 

The Navy and its prime contractors use receiving inspection information to 
grade sub -tier contractors and suppliers. 

CPARS provides source evaluation boards access to past performance for 
large contracts. 

To date, the RYG Program has withstood the scrutiny of courts during three 
legal challenges. 

3.6 

Certification Processes 


3.6.1 

New Design/Construction SUBSAFE Certification 

The SUBSAFE certification process involves a focused, independent audit 
organization (NAVSEA 92Q) to verify process capability and discipline. This 
same organization conducts detailed vertical audits of selected critical 
submarine components during certification audits. 

Multiple NAVSEA certification processes operate in parallel to ensure safety 
and operational readiness. 

REG (re-entry control) is invoked early in the certification process. 
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Summary of Key Observations 


3.6.2 Maintenance/Modernization SUBSAFE Certification 

Each individual project superintendent (dedicated 100% to work for a single 
availability) conducts advance planning for compliance with SUBSAFE 
requirements. This approach provides for focus, continuity, and clear 
accountability. 

The SUBSAFE certification process involves a focused, independent 
compliance verification organization (NAVSEA 92Q) to verify process 
capability and discipline. This same organization conducts vertical audits of 
selected critical submarine components during certification. 

Multiple NAVSEA certification processes operate in parallel to ensure safety 
and operational readiness. 

REC (re-entry control) is invoked as a vehicle to ensure that any work 
accomplished within the SUBSAFE boundary is planned, executed, and tested 
in a controlled and deliberate manner. 

3.7 

NAVSEA 08 Naval Reactors 


NAVSEA 08 relies on highly qualified, highly trained people who are held 
personally accountable for safety. 

The NAVSEA 08 management hierarchy (including support management, 
e.g.. Public Communications Director) is technically trained and qualified in 
nuclear engineering and experienced in nuclear reactor operating principles, 
requirements, and design. 

Problem awareness is essential to high degrees of safety and reliability. 
NAVSEA 08 demands that problems be raised to the proper levels when they 
happen and promotes the airing of minority opinions. 
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Appendix C: Team Membership 


NASA Review Team 


The NASA Office of Safety 
and Mission Assurance 
(OSMA) review team was 
led by Dr. J. Steven 
Newman, supported by core 
team members Mr. Stephen 
M. Wander, Mr. John 
Castellano. The NASA 
team also includes RADM 
Walter Cantrell USN (Ret.) 



Figure C.l NASA/Navy Benchmark Exchange Team: Tactical 
Planning Meeting August 15, 2002, Washington, DC. 

Photo M. Kowaleski 


and David Lengyel, 

Director of the NASA 
Aerospace Advisory 
Panel. Center-based core 
team members include Mr. Roy Malone and Mr. Vygantas Kulpa from Marshall Space 
Flight Center (MSFC), Mr. Randy Segert from KSC, and Mr. Michael Fodroci, from 
Johnson Space Center. NASA subject matter experts include Dr. Michael Stamatelatos 
(risk management), Ms. Faith Chandler (human factors), Ms. Martha Wetherholt and Ms. 
Alice Lee (software), and Mr. Tom Whitmeyer (quality). NASA Team executive 
secretary support was provided by Mr. Henry Hartt, BAE Systems and Mr. Don Vecellio, 
ARES Corporation. 


Navy Review Team 

The Navy Team was led by CDR Tom Van Petten and Mr. A1 Eord, and supported by 
Mr. Jim Eawrence, Mr. Brian Hughitt, and Mr. Angus Hendrick. Other key Navy 
participants included Mr. Steve Sites, Mr. Prank Tesoriero, Ms. Kathy Boyles, Mr. Steve 
Rotolo, Mr. Steve Southard, Mr. Eance Tracey, and Mr. Eric Snider. Navy Team 
logistics support was provided by Mr. Dan Rooney and Mr. Tom Bacus of Perot Systems. 


NNBE Management Team 

A management team was formed to direct the overall NNBE. This team is co- chaired by 
senior representatives from NASA's Office of Safety and Mission Assurance and the 
NAVSEA 92Q Submarine Safety and Quality Assurance Division. Principal NNBE 
management team members included Dr. J. Steven Newman, Mr. Stephen M. Wander, 
Mr. John Castellano, Mr. A1 Eord, Mr. Jim Eawrence, Mr. Brian Hughitt, Mr. Henry 
Hartt, Mr. Don Vecellio, Mr. Dan Rooney, and Mr. Tom Bacus. 
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Appendix D: Key Documents Reviewed/Exchanged 


NAVSEA HQ 

1. NAVSEA 0924-062-0010 Rev C, Submarine Safety (SUBSAFE) Requirements 
Manual, dated 5 Aug 02. (FOUO) 

2. NAVSEA SS800-AG-MAN-010/P-9290, Revision A, “System Certification 
Procedures and Criteria Manual for Deep Submergence Systems. 

3. “Summary of Activities of the NAVSEA Submarine Safety Senior Review Group 
from October 1986 thru Janary 1988,” September 1989. 

4. “SUBSAFE Program Organization,” presented at the Washington Navy Yard, August 
13, 2002. 

5. SUBSAFE Program Overview, presented by A1 Ford, Deputy Director, SUBSAFE 
and Quality Assurance, August 13, 2002. 

6. “Deep Submergence Systems Presentation,” by Mr. Steve Southard, NAVSEA, 
August 13, 2002. 

7. “Supervisors of Shipbuilding Conversion & Repair: DoD’s Shipbuilding & Ship 
Repair Technical and Contracts Expertise” presentation of August 13, 2002. 

8. “The Navy Ship Industrial Base,” slide presentation to the NASA/NAVY 
Benchmarking Team on August 13, 2002. 

9. “Submarine Personnel Qualification Program” presentation by Carl Haines, August 
13, 2002. 

10. “Generic Training and Educational Requirements” presentation of August 13, 2002. 

11. “Material Procurement” presentation of August 13, 2002. 

12. Memo dated 17 Jul 2002, from Commander, Pearl Harbor Naval Shipyard and 
Intermediate Maintenance Facility to Commanding Officer, USS TOPEKA, Subject: 
Certification Continuity of USS TOPEKA (SSN 754), Availability 201. 

13. Memo dated 30 Aug 2002, from Commander, Naval Sea Systems Command, Subject: 
Detailed Agenda for Submarine Safety Working Group (SSWG) Meeting of 17-19 
September 2002. 

14. NAVSEA INSTRUCTION 5400.95B, dated 01 May 2002, Subject: Naval Shipyard, 
SUPSHIP and Fleet Engineering and Technical Authority Policy. 
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15. NAVSEA INSTRUCTION 5450.55B, dated 18 Jun 01, Subject: Mission, Functions 
and Tasks of the Submarine Maintenance, Engineering, Planning and Procurement 
Activity (SUBMEPP), Portsmouth, NH. 

16. Memo dated 7 Aug 00, from Commander, Naval Sea Systems Command to 
Commanding Officer Submarine Maintenance Engineering, Planning and 
Procurement (SUBMEPP) Activity, Subject: Delegation of Technical and Approval 
Authority to SUBMEPP, with enclosure. 

17. Report dated 02 May 2002, SSN 688 Class Maintenance Plan Rev. Eist of All 
Effective Maintenance Requirements. 

18. SAMPEE CMP DOCUMENT: “SSN688 Class Maintenance Plan Rev - Eist of All 
Effective Maintenance Requirements,” 02 May 2002. (FOUO) 

19. SAMPEE AP DOCUMENTS: “SSN688 Class Maintenance Requirements (SER 
109-01), Ship System Work Description, SWEIN 708A01,” Issue 05 Rev 06, p. 3 of 
6, Nov 2000. (FOUO) 

20. “URO CHANGE: 116 Jun 2002 - Periodic Maintenance Requirements Inventory 
URO/HIP Sorted by System” Report Date: 30 Jul 2002. (NO FORN) 

21. “USS City of Corpus Christi (SSN705) ERO-1 CHGS 311 WP, Ship System Work 
Description - Issue Two (Re-issue), SWEIN 708A01,” p. 4 of 10. (NO FORN) 

22. “USS City Corpus Christi (SSN 705) REN 01 Work Package Supplement, Shipyard 
Planning & Feedback Report,” OWP Issue 02, URO Change No. 097, SWEIN 
708A01 - Part 1, 27 Dec 01. (NO FORN) 

23. SAMPEE MID DOCUMENTS: SUBMEPP MS No. 5510-081-073 Rev C Chg-2, 
“Submarine Maintenance Standard, MR Description: Restore Torpedo Tube Blow 
and Vent Isolation Ball Valves (AEP-274, 275, 276, and 277), Class: SSN688” 
05/17/02. (FOUO) 

24. SUBMEPP MS No. 7650-081-001 Rev G, “Submarine Maintenance Standard, MR 
Description: General Acceptance Criteria, Class: SSN21, SSBN726, SSN688, 
SSN683, MTS 626, 635, SSN774” 02/16/01. (FOUO) 

25. URO MRC 008 MOD 40, “Unrestricted Operation (URO) Maintenance Requirement 
Card (MRC) URO MRC 008 MOD 40,” Naval Sea Systems Command, p. 1 of 63, 
September 2002/Change 117. 

26. “SWEIN -GCM A- Description Database Printout,” p. 96 of 106, June 05, 2002. 
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27. “SUBSAFE - SUBMEPP Standardized Test Procedure, Test No: 708-2-1459, 
Rev/Chg: E2, Class: SSN688, Title: MK 67 Torpedo Tubes - Strength and Tightness 
Test,” 03/27/02. 

28. “SUBSAEE - SUBMEPP Standardized Test Procedure, Test No: 708-5-1466, 
Rev/Chg: D, Class: SSN688, Title: MK 67 Torpedo Tube Water Slug Eiring 
(Submerged),” 02/08/02. 

29. NAVSEA Viewgraph presentation regarding “Class Maintenance Planning.” 

30. NAVSEA Viewgraph of SUBMEPP Supported BPMP Milestones (Plow Chart). 

31. NAVSEA Viewgraph of Availability Work Package Planning Process. 

32. SUBMEPP Reliability Centered Maintenance (RCM) Analysis Risk Assessment 
Analysis, RCM Number: 55800902 Rev 0, printed 17-Oct-02. (POUO) 

33. SUBMEPP Command Overview, dated Oct 21, 2002. 

34. Baseline AIM “Overview of Component Unit Identifiers,” Work Packaging & 
Control (WPC) Process (Rev. 9311). 

35. Uniform Industrial Process Instruction, “Critique Process; Problem Identification and 
Investigations,” issued 05/30/02, Norfolk Naval Shipyard. (POUO) 

36. Submarine Maintenance Standard MS NO. 2560-081-047 REV C CHG-2, “MR 
Description: Restore main seawater hull valve hydraulic actuators,” dated 9/21/01. 
(NOPORN) 

37. SUBSAEE SUBMEPP Standardized Procedure, Test No. 714-3-1477 and DA-3- 
595 A- 1477, REV/CHG: E4, CEASS: SSN688 and SSBN726, “3-Inch Eauncher - 
Grooming,” dated 9/3/02. 

ELECTRIC BOAT 

38. Presentation Slides - “System Safety Electric Boat Corporation.” 

39. Presentation Slides - “Materials Management Overview Presented to NASA,” Blair 
Decker, October 17, 2002. 

40. Presentation Slides - "Reliability & Maintainability - Dept 408 RMA and System 
Safety Engineering - Supervisor: Steve Eibby. 

41. Presentation Slides - “Establishment and Maintenance of Supplier Ratings.” 

42. Presentation Slides - “Risk Management - The Electric Boat Sat-Por-Sea Process.” 
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43. Presentation Slides - “President's Staff’ and other organization charts. 


44. Presentation Slides - “Safety Requirements - Definition and Management Emphasis 
Flowdown to Subcontractors: QA Requirements, Software Assurance 
Requirements).” 


SUBMEPP 

45. “SUBMEPP Command Overview” Presentation by Capt Herbein, October 21, 2002. 

46. “Delegation of Technical and Approval Authority to SUBMEPP” COM NAVSEA 
letter 91 10 - Ser TSUB/213 - 7 Aug 00. 

47. Portsmouth Naval Shipyard Organization Trees. 

48. Submarine Maintenance Standard - MS NO. 2560-081-047 Rev C Change-2 9/21/01 
(“Restore main sea water hull valve hydraulic actuators”). (NOFORN) 

49. NAVSEA Instruction 5450. 55B, “Mission, Eunctions and Tasks of the Submarine 
Maintenance, Engineering, Planning and Procurement Activity (SUBMEPP), 
Portsmouth, NH” NAVSEAINST 5450.55B - Ser 92/015 - 18 June 01. 

50. NAVSEA Instruction 5400.95B, “Naval Shipyard, SUPSHIP and Fleet Engineering 
and Technical Authority Policy,” NAVSEAINST 5400.95B - Ser 05BX/004 - 01 
May 2002. 

51. “3 Inch Eauncher - Grooming” -- SUBMEPP Standardized Test Procedure 
“SUBSAFE” Test No 714-3- 1477, DA-3-595A-I477, Rev E4, SSN688/SSBN726 
Class - 9/3/2002 (DoD/DoN distribution only without further authorization). 

(FOUO) 

52. “SUBMEPP Reliability Centered Maintenance Handbook,” SUBMEPP 9081-083- 
0589, 4 December 1995. 

53. SUBMEPP Reliability Centered Maintenance (RCM) Analysis -- Risk Assessment 
Analysis. (FOUO) 

54. “Unrestricted Operation (URO)/Maintenance Requirement Card (MRC), URO/MRC 
006 Mod 40, Inspect ASW and MSW Pump Casing Wall Thickness” September 
2002/Change 117. 

55. “SUBMEPP Supported BPMP Milestones” - Chart. 
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56 . 


57. 

58. 


59. 

60. 


61. 

62. 

63. 


64. 


65. 

66 . 


67. 

68 . 

69. 


“Ckss Maintenance Planning,” “Availability Planning,” “Material Support Process,” 
“Maintenance Instruction Documents,” and “Engineering Process” -- Series of 
Viewgraph pitches. 

"Index Of Naval Shipyard Standard Process Instructions," Issue No.74_NAVSEA 
Technical Information Support Office, March 2001. 

Advanced Industrial Management (AIM) presentation. 

NAVSEALOGCEN 

Prime Contractor Partnership Program brochure. 

Presentation: “Tools for Source Selection, CPARS - Contractor Performance 
Assessment Reporting System /PPIRS Past Performance Information Retrieval 
System,” Ed Marceau, CPARS Program Manager, Naval Sea Eogistics Center. 

Elyer: "Navy Air Eorce Red/Yellow/Green Program." 

Sample PDREP report. 

“VISION DoD PAST PEREORMANCE INEORMATION (Collectbn and 
Retrieval)”, Skip Smith May 7, 2002 Presentation at 5^’’ Annual Procurement Quality 
Conference. 

Best Manufacturing Practices blue book on NAVSEAEOGCEN -- “REPORT OP 
SURVEY CONDUCTED AT NAVAE SEA EOGISTICS CENTER 
DETACHMENT PORTSMOUTH, NH JUEY 2000,” Best Manufacturing Practices 
Center of Excellence, College Park, MD. 

Product Data Reporting and Evaluation Program (PDREP), SECNAVINST 4855. 3A, 
ASN (RDA) 9 July 1998. 

Sample “Product Data Reporting and Evaluation Program Contractor Profile Run, 22- 
Oct-02.” 


SHAPEC 

“Plow of SHAPEC Action Requests” - Chart. 
“SHAPEC 101 - Job Summary Development” - Chart. 
“Availability Work Package Planning Process” - Chart. 


70. "Ship Availability Planning and Engineering Center (SHAPEC) Operations Manual," 
2 July 1998 and 2002. (DRAPT) 
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71. "SSN688 Class Ship Availability Planning & Engineering Center (SHAPEC) Process 
Manual," Revision A Change 1, April 2001. 

NAVSEA 08 

72. “The United States Naval Nuclear Propulsion Program,” published jointly by the 
Department of Energy and The Department of the Navy, March 2002. 

73. Comments by Admiral H. G. Rickover, USN, Director, Naval Nuclear Propulsion 
Program, Subsequent to the Accident at Three Mile Island, August 1979, 

“Differences Between Naval Reactor and Commercial Nuclear Plants.” 

74. Viewgraph presentation, “United States Naval Nuclear Propulsion Program.” 

75. Statement of Admiral H. G. Rickover, USN, Before the Subcommittee on Energy 
Research and Production of the Committee on Science and Technology, U.S. House 
of Representatives, May 24, 1979. 
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Appendix E: NNBE Acronyms and Terms 


ADS 

Atmospheric Dive Suit 

APU 

Auxiliary Power Unit 

ARC 

Ames Research Center 

ASAP 

Aerospace Safety Advisory Panel 

Availability 

Availability of ship/depot for required maintenance 

BESS 

Basic Enlisted Submarine School 

CA 

Corrective Action 

CAR 

Corrective Action Report 

CASREP 

Casualty Report 

CBM 

Condition- Based Maintenance 

CCB 

Change Control Board 

CDR 

Critical Design Review 

CIE 

Critical Items Eist 

CECS 

Computer Eaunch Control System 

CES 

Contingency Eanding Site 

CM 

Configuration Management 

CoER 

Certification of Elight Readiness 

COMSUBEANT 

Commander Submarines, Atlantic 

COMSUBPAC 

Commander Submarines, Pacific 

DCAA 

Defense Contract Audit Agency 

DDS 

Dry Deck Shelter 

DIP 

Design Improvement Proposal 

DSRV 

Deep Submer^nce Rescue Vehicle 

DSS-SOC 

Deep Submergence System Scope of Certification 

ECP 

Engineering Change Proposal 

EQA 

Engineering Quality Assurance 

ET 

External Tank 

EDRD 

Elight Definition Requirements Document 

EMEA 

Eailure Modes and Effects Analysis 

ER 

Eiring Room 

ERR 

Elight Readiness Review 

ETA 

Eault Tree Analysis 

GD/EB 

General Dynamics/Electric Boat 

GMIP 

Government Mandatory Inspection Point 

GPC 

General Purpose Computer 

GRC 

Glenn Research Center 

GSEC 

Goddard Space Plight Center 

GSI 

Government Source Inspection 

HM&E 

Hull, Mechanical and Electrical 

IMA 

Intermediate Maintenance Activity 

INSRP 

Interagency Nuclear Safety Review Panel 

IPT 

Integrated Product Team 

ISE 

In-Service Engineer 
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IV&V 

Independent Verification and Validation (software) 

JFMM 

Joint Elect Maintenance Manual 

JPL 

Jet Propulsion Eaboratory 

JSC 

Johnson Space Center 

KSC 

Kennedy Space Center 

LAR 

Eiaison Action Report 

LaRC 

Eangley Research Center 

LARS 

Eaunch and Recovery System 

LH2 

Eiquid Hydrogen 

LI 

Eevel I 

LOX 

Eiquid Oxygen 

L-X 

X days to launch 

MAF 

Michoud Assembly Eacility 

MAT 

Major Area Team 

MAXQ 

maximum dynamic pressure 

MECO 

Main Engine Cutoff 

MET 

Mission Elapsed Time 

MGT 

Management 

MIC 

Mission Integration Center 

MEP 

Mobile Eaunch Platform 

MMP 

Multi- Mission Platform 

MMT 

Mission Management Team 

MOA 

Memorandum of Agreement 

MRB 

Material Review Board 

MRC 

Maintenance Requirement Card 

MRR 

Mission Readiness Review 

MSEC 

Marshall Space Plight Center 

MSRP 

Material Selection Requirements Plan 

NAVAIR 

Naval Air Systems Command 

NAVSEA 03: 

Human Systems Integration in System Design 

NAVSEA 04 

Eogistics, Maintenance, & Industrial Operations 

NAVSEA 05 

Ship Design, Integration and Engineering 

NAVSEA 08 

Naval Reactors 

NAVSEA 92 

Submarine Directorate 

NAVSEA 92Q 

SUBSAFE/Quality Assurance 

NAVSEA 92T 

Technical 

NAVSEA 

Naval Sea Systems Command 

NAVSEAEOGCEN 

NAVSEA Eogistics Center 

NDE 

Nondestructive Evaluation 

NDT 

Non- Destructive Testing 

NEPA 

National Environmental Policy Act 

NEQA 

NASA Engineering and Quality Audit 

NNBE 

NASA/Navy Benchmarking Exchange 

NNPI 

Nuclear Navy Propulsion Information 

NPD 

NASA Policy Directive 

NPG 

NASA Procedures and Guidelines 
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NPS 

Nominal Pipe Size 

NR 

NAVSEA 08 Naval Reactors 

NRRO 

Naval Reactors Representative Office 

NSWC 

Naval Surface Warfare Center 

NUWC 

Naval Underwater Warfare Center 

OMI 

Operations Maintenance Instruction 

OPF 

Orbiter Processing Eacility 

OPNAV 

Office of Chief of Naval Operations 

OQE 

Objective Quality Evidence 

ORD 

Operational Requirements Document 

P/L 

Payload 

PAR 

Pre- flight Assessment Review 

PAT 

Process Action Team 

PDR 

Preliminary Design Review 

PE 

Procedures Evaluation 

PEO 

Program Executive Officer 

PEMEA 

Process Eailure Modes and Effects Analysis 

PM 

Program Manager 

PMAG 

Program Management Assistance Group 

PMMT 

Pre- launch Mission Management Team 

PNS 

Portsmouth Naval Ship Yard 

PQA 

Procurement Quality Assurance 

PR 

Procedures Review 

PRA 

Probabilistic Risk Assessment 

PRCB 

Program Requirements Control Board 

PV 

Process Verification 

PVI 

Product Verification Inspection 

QA 

Quality Assurance 

QAEI 

Quality Assurance Eetter of Inspection 

QDR 

Quality Deficiency Report 

QPM 

Quality Performance Manual 

RCM 

Reliability- Centered Maintenance 

REC 

Re-entry Control 

REI 

Ready Eor Issue 

RTES 

Return to Eaunch Site 

SAIE 

Shuttle Avionics Integration Eaboratory 

SAP 

Ship Alteration Proposal 

SAR 

Ship Alteration Record 

SCA 

Shuttle Carrier Aircraft 

SCN 

Ship Construction, New 

SHAPEC 

Ship Availability Planning and Engineering Center 

SHIPAET 

Ship Alteration 

SIMA 

Shore Intermediate Maintenance Activity 

SIT 

System Integration Team 

SEI 

Space Eaunch Initiative 

SOE 

Special Operations Eorces 
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SOM 

SUPSHIP Operations Manual 

SOSG 

Supervisor of Shipbuilding, Groton 

SOSN 

Supervisor of Shipbuilding, Newport News 

SOW 

Statement of Work 

SRB 

Solid Rocket Booster 

SRC 

Submarine Rescue Chamber 

SRD 

Selected Record Drawing 

SS 

SUBSAEE 

SS Manual 

NAVSEA 0924-062-0010 Rev C 

SSBN 

Submarine Ship, Ballistic, Nuclear 

SSC 

Stennis Space Center 

SSGN 

Submarine Ship, Guided, Nuclear 

SSIC 

SUBSAEE Improvement Committee 

SSME 

Space Shuttle Main Engine 

SSN 

Submarine Ship, Nuclear (East Attack) 

SSPD 

SUBSAEE Program Director 

SSWG 

SUBSAEE Working Group 

STS 

Space Transportation System (Space Shuttle) 

SUBMEPP 

Submarine Maintenance Engineering, Planning and Procurement 

SUBSAEE 

Submarine Safety 

SUPSHIP 

Supervisor of Shipbuilding, Conversion and Repair 

TAG 

Technical Advisory Group 

TAE 

Trans -Atlantic Eanding 

TCDT 

Terminal Countdown Demonstration Test 

TOP 

Technical Data Package 

TECH 

Technical 

TGI 

Task Group Instruction 

TRS 

Training Record System 

TWD 

Task Work Document 

TYCO 

Type Commander (COMSUBEANT and COMSUBPAC) 

UMR 

Unsatisfactory Material Reporting 

URO 

Unrestricted Operations 

UTRS 

Uniform Technical Requirements System 

VAB 

Vehicle Assembly Building 

WAD 

Work Authorization Document 

WAE 

Work Authorization Eorm 

WBS 

Work Breakdown Structure 

WNY 

Washington Navy Yard 

WSESRB 

NAVSEA Weapons System Explosive Safety Review 


Board 


i 


NNBE INTERIM REPORT 


85 


